Back in May I archived the Pirate101 Subreddit in the midst of Reddit’s API shutdown. This was necessary to grab content created up to that point, but I would need to periodically repeat the process manually in order to grab new content. There is no way I would remember to do this on a regular …
As I briefly mentioned in last month’s post, I recently built a new computer which uses Linux as its operating system (OS). This means improvements in speed, security, and privacy, but at the cost of program compatibility. Where in a Windows or Mac environment most things “just work”, Linux can require a little bit of …
Nearly every script I have written for this project has been in Python. And for good reason, too: there are lots of pre-made modules that make certain tasks simpler, Python is easy and fast to write code in, and there are tons of examples of programs online that I can base things off of. The …
It’s bit late to start doing this but here goes. Like so many other content providing services (recently Twitter and Imgur), Reddit recently made the decision to stop offering its powerful free API. It then went a step further and demanded that services which publicly hosted searchable archives stop providing access. A third party service …
Discord is a great platform. If for some reason you have not had the opportunity to interact with it, I would suggest checking it out. Discord allows for free, instant voice and text chat with a large amount of people at once. Each “server” is usually dedicated to one topic or run by one organization …
As I dug through my scraping logs, I encountered something I didn’t want to see: lots of logged errors. This is particularly fitting because I created the auto-scraping mechanism last March, approximately one year ago. Final Bastion is the site in question and the errors had existed more or less since the beginning of the …
Last month’s post was about the background of Google+ and a preliminary attempt to scrape the limited data that was gathered before the shutdown. I wrote a script to read through all of the .cdx.idx files but this did not yield any results. So I decided to bite the bullet and start scraping the full …
In the 2022 recap post I made earlier this month one of the goals I set was to scrape Google+ for its Pirate101 data. This has been a lofty goal of mine since I first started this projected in August 2021 and, since this is my last semester with the essentially unlimited bandwidth of my …
I started this project in April 2021 but didn’t get around to making a website and tracking my progress until much later that same year. Given that 2022 was the first full year of the .P101 network I thought I would look over what I accomplished and declare what I want to work on in …
This is the reminder that you didn’t know you needed to back up your important data and verify that it works! The cloud hosting provider that I had been using decided to delete my VPS and their support is unresponsive. I have since moved to a new hosting company that has a much better track …