HOME·ANSWERS·XERJ AUTOINDEX RECORDED MY TAKEOUT .ZIP AS JUNK AND SAID TO EXTRACT IT FIRST. WHY?
ANSWERS · OPERATIONS: EXIT CODES

xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?

AUTHOR · XERJ documentation team · REVIEWED BY · XERJ engineering team · PUBLISHED · 2026-09-18 · UPDATED · 2026-09-20

TL;DR — autoindex does not open archives. When it meets a .zip, .tgz, .7z, .rar or .xz, it marks the file as junk with a reason that quotes the extraction command, and prints that under the run's summary. The run finishes with exit code 3 and the rest of the folder is indexed. Extract the archive and run the same command on the extracted folder.

PASTE THIS TO YOUR AI AGENT — IT DOES THE REST Act as a coding agent. Read https://xerj.org/llms.txt. If a run prints 'not indexed — archives are never opened' under its summary (or its --json summary has a non-empty unextracted_archives list), run the command each line names (unzip, tar -xzf, 7z x, unrar x, xz -dk) on that file, then run xerj autoindex again on the extracted folder. Exit code 3 with those lines is a completed run, not a failure, but if the archive was the only thing in the folder nothing was indexed: tell the user.
RUN THIS XERJ COMMAND xerj autoindex ~/downloads --url http://127.0.0.1:9200 --prefix dl --progress plain A run over a folder that holds an archive finishes with exit 3 and prints the archive, with the command to extract it, under its summary.
RUN THIS XERJ COMMAND bash -c 'unzip takeout-20260101T000000Z-001.zip -d ~/mail-export' Do what the reason says: extract the archive next to nothing you will index twice.
RUN THIS XERJ COMMAND xerj autoindex ~/mail-export --url http://127.0.0.1:9200 --prefix mail --progress plain Index the extracted folder. Its mbox, notes and documents are sniffed by content.

What the run prints

Under its summary, the run lists every unextracted archive with its reason:

not indexed — archives are never opened; extract, then run this command on the extracted folder:
  takeout-20260101T000000Z-001.zip — unextracted zip archive: autoindex does not open archives — extract it first (unzip <file>), then run autoindex on the extracted folder

The same lines are in the --json summary as unextracted_archives. After ten archives the list stops and says how many more there are; xerj autoindex map lists every one under "Junk / skipped".

The kind and the command change with the bytes. A tar gets tar -xf, a gzip-compressed tar tar -xzf, a 7z 7z x, a rar unrar x, an xz file xz -dk. Detection is by the file's leading bytes. A tar is recognised by its header checksum, not by the word ustar appearing in a text file. So a renamed archive gets the same treatment.

Why it does not just open the archive

Two reasons. Extracting is one command you already have, and a reason that quotes it costs nothing. And indexing inside archives is the way a folder ends up indexed twice: once from the archive's contents and once from the extracted copy beside it. Every document would then have two ids. Refusing keeps the index honest.

What to do

unzip takeout-20260101T000000Z-001.zip -d ~/mail-export
xerj autoindex ~/mail-export --url http://127.0.0.1:9200 --prefix mail --progress plain

For a Google Takeout export, the extracted folder holds the mailbox. It is detected by content and streamed. The Takeout article covers what comes back. A plain .gz that is not a tar is not an archive in this sense. It is decompressed on the fly and sniffed like the file inside.

FAQ

Why does xerj autoindex say to extract a zip first?

Because it never opens archives. Extracting is one command it quotes in the reason, and it keeps the index from holding the same files twice — once inside the archive and once beside it.

Is the run a failure?

No. The archive is recorded as junk with its reason and everything else in the folder is indexed. The exit code is 3, completed-with-junk. If the archive was the only file, the run indexed nothing, and the lines it prints under its summary say why.

Which archive kinds does it recognise?

zip, tar and gzip-compressed tar, 7z, rar and xz. Each is detected by its bytes, and the reason names the right command for that kind.

What about a .gz that is not a tar?

A plain gzip-compressed file is streamed through decompression and sniffed like the file inside; only a gzip that contains a tar is treated as an archive.

Evidence

Related