Internet Archive CLI Tool

Start Page Browser Play! Minetest MT worlds Bucket Game Player Basics Downloads Contact Devel Center

minetest.org minetest.io 9,000 gits CoderChat EdgyNet

---

Introduction

The Internet Archive can be used for the distribution and backup of FOSS code and data. Warning: The procedures involved are slow, finicky, flaky, and unreliable. However, if this approach still works as of the current date, it will be useful to some extent.

In particular: As of Fall 2023, Internet Archive was willing to host Minetest world tarballs that were gigabytes in size.

We recommend adding screenshots and descriptions and other metadata or the nature of the uploads may not be clear and Internet Archive may block accounts.

This page provides some information related to an Internet Archive CLI tool.

Internet Archive CLI tool 

The steps below worked as of Fall 2023. They may or may not still work.

S1) As of Fall 2023, the link below worked:

https://ia800702.us.archive.org/27/items/ia-pex/ia

The link pointed to a Python 3 binary script. The script was compatible with Debian 11. It may or may not work with older or newer Debians.

The script did not work with a non-Debian distro named Laclin.

As a last resort, if "ia" doesn't work with your Linux distro, you should be able to use it by running it under Debian 11 in a "podman" container.

Install the script on a Debian 11 box [or container] and chmod it to 755. This might then provide CLI access to Internet Archive.

S2) Use the following command to configure the tool:

ia configure

You'll be asked to specify the credentials for the Internet Archive account that you'd like to operate on. The credentials are the associated email address and password.

Out of the box, "ia" only supports one IArchive account at a time. However, you can use it to work on multiple accounts by creating and preserving multiple "ia" configuration files, one per IArchive account. Details are to be added.

S3) Subsequent to the "configure" step, a command similar to the following may work to upload a source code tarball to Internet Archive:

ia upload \
--metadata=mediatype:software \
23112301 \
minetest-assemble-231123.tgz

The user needs to enter some information manually using the Internet Archive web GUI. This includes title, original creator, original date, description, and keywords.

The "23112301" string in this example should be replaced with a unique item ID. Note: The ID can't be changed later. The ID can be a descriptive string [using underscores instead of spaces] but for technical reasons this isn't recommended.

S4) To replace the uploaded file, without deleting the item ID, one could try this:

ia delete 23112301
# Repeat the "ia" upload

Replace 23112301 here with the appropriate item ID. Note: This operation deletes the contents of the item ID but not the item ID itself.

S5) Three warnings:

Warning #1: If one deletes an item ID at the GUI level, it is then gone forever.

Warning #2: It isn't possible to change mediatype for an item ID once it's set. This doesn't work by way of either the IArchive GUI or the CLI tool.

An item ID can have multiple files. The primary file needs to be uploaded first and the mediatype needs to be specified, and specified correctly, for that file.

This is an unusual limitation but it does seem to be the base. As of Fall 2023, there were numerous posts in an IArchive support forum about the issue.

Warning #3: "ia" operations don't complete immediately on the Internet Archive side. Sometimes they may never complete. The only way to clean things up may be to delete an item ID on the GUI side.

The third warning above is one reason to use numeric IDs instead of descriptive IDs. If you use a descriptive ID, you may end up needing to delete it and it won't be possible in this case to bring it back. So descriptive IDs may end up being lost permanently and therefore wasted.

S6) Supported mediatypes include: software, movies, image

"text" or "texts" may work as well but this hasn't been confirmed yet.

If the mediatype is omitted, it should default to "other".

S7) If a source-code tarball is uploaded in this manner, one can upload a JPEG or PNG file subsequently to the same item ID:

ia upload 23112301 screenshot.png

Note that mediatype isn't specified here because this isn't an upload of the primary file.

If things work correctly, the IArchive GUI may then display the screenshot in connection with the tarball.

End of page