The internet is disappearing, so I repurposed an old laptop to save it


Every day, parts of the web are disappearing. Content that isn’t taken down is being edited and paywalled. What can you do about it? You can install this open source server and browser extension, which is what I did.

Archiving media is one of the most time-honored methods of preservation. You might have heard of the Wayback Machine, which is a service run by the non-profit called the Internet Archive. It lets you retrieve old versions of websites, plus websites that don’t exist at all anymore, thanks to people who archived it when it was still live. Knowledge is power, and the Wayback Machine isn’t the only way to collect and review historical knowledge of online content.

ArchiveBox is my own little Wayback Machine

There’s no place like local storage

The ArchiveBox web interface showing three websites saved.

You can use the Wayback Machine browser extension to request a web page be archived. But what if you could host a private version of that service on your own hardware?

The open source project ArchiveBox works similar to the Wayback Machine, but you host it on your own machine, and the content you archive get saved directly to your storage so you have complete control of it. Any website you come across, you can send a request to your ArchiveBox server, and it will attempt to save copies of it in multiple formats, including in HTML, as a PDF, a screenshot, and more. It can even pull save certain types of embedded videos.

A monitor with the browser open and a message indicating that the computer is offline


How to Copy a Whole Website to Your Computer Using wget

Want to save an entire webpage to your PC? No problem.

I was excited to find ArchiveBox because I often come across useful information published online that I get concerned won’t be publicly available for long. The common saying “the internet is forever” isn’t as true as it feels.

I could send content I want saved to the Wayback Machine, but it’s also not clear that the Internet Archive will be around forever. Internet access, even in the United States, is not a guarantee, either. If I want to be sure something gets saved and that I don’t lose possession of it, my own personal ArchiveBox can guarantee that.

How I installed ArchiveBox

An automated bash script saved me a lot of time

Linux terminal open in the Ubuntu desktop on a laptop screen. Credit: Hannah Stryker / How-To Geek

There are several ways you can install ArchiveBox. You can run the service directly on your desktop computer, or you can host it on a dedicated device and connect to it over your network.

Personally, I had an old laptop I’d installed a lightweight Linux distribution on but wasn’t using. I decided I’d repurpose it as my archive center since it had plenty of unused storage. It was a very outdated MacBook, so the hardware wasn’t impressive, but for web crawling work like ArchiveBox does, that’s not much of a problem.

Once you’ve chosen a host, it’s time to actually install ArchiveBox. One of the easiest methods, and the one I used, was to just run this Bash script in Mac, Linux, or WSL, which installs and configures a basic Docker container with all of ArchiveBox’s required tools and services running in it.

curl -fsSL 'https://get.archivebox.io' | bash

Once the script completes, all you need to do is open a web browser and type localhost:8000 into your address bar. Or, if it’s hosted on a remote machine, replace localhost with the device’s IP address.

You should get a web interface for ArchiveBox. You’ll want to bookmark the page because here is where you’ll find a web interface for managing your archive and reviewing content you’ve saved there. The first time you visit it, you’ll need to set up an admin account. You can also make other accounts if you want to share access to your archive with others.

ArchiveBox app icon.

OS

Windows, Mac, Linux

Key highlights

Open Source

ArchiveBox is an open source server solution that lets you archive the web as you explore it. Host it on a local device, get the browser extension, and you’ll never lose media and webpages from the internet you love.


The web extension is what makes it great

Archiving web pages becomes a one-click job

So once you have an ArchiveBox set up, how do you use it? Well, you could open a terminal and send commands like archivebox add --depth=1 'https://howtogeek.com'.

Personally, I didn’t want to mess with the command line while I was browsing the web. Fortunately, ArchiveBox has a dedicated extension for web browsers called ArchiveBox Exporter you can get for Chrome (and Edge) and Firefox. (For Safari users, there’s a shortcut you can set up that does the same job).

After installing the extension, all I needed to do was give it the address of my ArchiveBox server, and from then on I was able to archive at light speed.

The ArchiveBox Exporter extension settings in Firefox with a red box around the server address field.

The extension has two modes: Allowlist and BlockList. In Allowlist mode, it won’t archive anything unless you specify a web domain (like howtogeek.com) as ones you “allow” to be automatically archived by ArchiveBox when you visit them. You can also right-click any page, and in the context menu there’s an ArchiveBox menu of options that includes “Archive Current Page.”

A context menu open on a web article with a red box around the option to use ArchiveBox to save archive the current page.

Click over to your ArchiveBox web interface, and you’ll see the snapshot of the page being saved there.

The ArchiveBox web interface showing four websites saved.

In the much more aggressive Blocklist mode, the extension will send requests to ArchiveBox to capture every single web page you visit except for the domains you specify in the blocklist. I don’t recommend turning this on unless you’re sure what you’re doing, and if you’re not concerned with risks like running out of storage space.

Internet-preserving apps like Kiwix are also at your disposal

ArchiveBox is not the only way to download the internet to your personal storage. In fact, for saving entire websites themselves, like Wikipedia or iFixIt’s library of instructions, I recommend another app called Kiwix. With Kiwix, you can download prepared versions of popular websites and view them in one place. Export it to an external hard drive, and you have a compendium of information that can’t be taken from you by internet outages.

A browser window with a download button being clicked, download icons, and Kiwix bird logos.


How to download and browse whole websites offline with ZIM files

Imagine downloading the entire Wikipedia and browsing it offline, just like the regular version.



Source link

Leave a Reply

Subscribe to Our Newsletter

Get our latest articles delivered straight to your inbox. No spam, we promise.

Recent Reviews


Spotify aims to provide a consistent listening experience that uses minimal data. As a result, your audio quality might be less than ideal, especially if you’re using a pair of high-fidelity headphones or high-end speakers. Here’s how to fix that.

Switch audio streaming quality to Very High or Lossless

The default audio streaming quality in both the mobile and desktop Spotify apps is set to Automatic, which usually keeps the audio quality at Normal, which is only 96 Kbps. Even though Spotify uses the Ogg Vorbis codec, which is superior to MP3, OGG files exhibit slight (but noticeable) digital noise, poor bass detail, dull treble, and a narrow soundstage at 96 Kbps.

Even worse, Spotify is aggressive about adjusting the automatic bitrate. Even though 4G is more than fast enough to stream high-quality OGG files, even with a weak signal, Spotify may still drop the quality to Low, which has a bitrate of just 24 Kb/s. You will notice such a sharp drop in quality, even on a pair of bottom-of-the-barrel headphones.

To rectify this, open the Spotify app, tap your user image, open “Settings and privacy,” and tap the “Media Quality” menu. Once there, set Wi-Fi streaming quality and cellular streaming quality to “Very high” or “Lossless.”

I recommend setting cellular streaming quality to Very high and reserving Lossless for Wi-Fi, since lossless streaming is very data-intensive. One hour of streaming lossless files can take up to 1GB of data, as well as a good chunk of your phone’s storage, because Spotify caches files you’re frequently streaming. Besides, you’ll struggle to notice the difference unless you’re listening to music on a wired pair of high-end headphones or speakers; wireless connection just doesn’t have the bandwidth needed to convey the full fidelity of Spotify lossless audio.

You might opt for High quality if you have a capped data plan, but I recommend doing so only if you stream hours upon hours’ worth of music every single day over a cellular network. For instance, I burn through about 8 GB of data per month on average while streaming about two hours of very high-quality music over a cellular network each day.

Illustration of a headphone with various music icons around.


How Audio Compression Works and Why It Can Affect Your Music Quality

Feeling the squeeze when listening to your favorite song?

Set audio download quality to Very high or Lossless

If you tend to download songs and albums for offline listening, you should also set the audio download quality to “Very high” or “Lossless.” This setting is located just under the audio streaming quality section.

The audio download quality menu in Spotify's mobile app.

If you’ve got enough free storage on your phone, opt for the latter, but if you’d rather save storage space, set it to Very high. You’ll hardly hear the difference, but lossless files are about five times larger than the 320 Kb/s OGG files Spotify offers at its Very high quality setting, and they can quickly fill up your phone’s storage.

Adjust video streaming quality at your discretion

The last section of the Media quality menu is Video streaming quality. This sets the quality of video podcasts and music videos available for certain songs. Since I care about neither, I set it to “Very high” on Wi-Fi and “Normal” on cellular, but you should tweak the two options at your discretion because songs sound notably better at higher video streaming quality levels.

If you often watch videos over cellular and have unlimited data, feel free to toggle video quality to very high.

Make sure Data Saver mode is disabled

Even if your audio quality is set to Very high or Lossless, Spotify will switch to low-quality streaming if the app’s Data saver mode is enabled. This option is located in the Data saving and offline menu. Open the menu, then set it to “Always off,” or choose “Automatic” to have Spotify’s Data Saver mode kick in alongside your phone’s Data Saver mode.

You can also enable volume normalization and play around with the built-in equalizer

Spotify logo in the center of the screen with an equalizer in front. Credit: Lucas Gouveia / How-To Geek

Last but not least, there are two additional features you can play with to improve your listening experience. The first is volume normalization, which sets the same loudness for every track you’re listening to. This can be handy because different albums are mastered at different loudness levels, with newer music usually being louder.

Since I’m an album-oriented listener, I keep the option disabled. I can just play an album and set the audio volume accordingly, and I don’t really mind louder songs when listening to playlists, artists, or song radios.

But if you can’t stand one song being quiet and the next rattling the windows, visit the Playback menu, enable “Volume normalization,” and set it to “Quiet” or “Normal.” The “Loud” option can digitally compress files, and neither Spotify nor I recommend using it. This also happens with “Quiet” and “Normal,” since both adjust the decibel level of the master recording for each song, but the compression level is much lower and extremely hard to notice.

Before I end this, I should also mention that you can access the equalizer directly from the Spotify app, where you can fine-tune your music listening experience or pick one of the available equalizer presets. If your phone has a built-in equalizer, Spotify will open it; if it doesn’t, you can use Spotify’s. On my phone (a Samsung Galaxy S21 FE), I can only use One UI’s built-in equalizer.

To open the equalizer, open “Playback,” then hit the “Equalizer” button. Now you can equalize your audio to your heart’s content.


Adjusting just a few settings can have a drastic impact on your Spotify listening experience. If you aren’t satisfied with Spotify’s sound quality, make sure to adjust the audio before jumping ship. You should also check the sound quality settings from time to time, as Spotify can reset them during app updates.​​​​​​​

Three phones with a Spotify screen and the logo in the center.


These 8 Spotify Features Are My Favorite Hidden Gems

Look for these now.



Source link