Automating My Now Page, Update

Although I’ve written previously (first here, then here) about how I (semi-)automated my Now page, I’ve made some substantial changes to the code and the underlying database recently, so I thought I should do a refresh.

Why automate in the first place?

The omg.lol ecosystem makes it very easy to maintain my web content, so why not just use the tools available? The biggest reason for me is to be able to keep a record of what’s been on my now page over time. The other reason, if I’m being totally honest, is the challenge and fun of it.

Why not automate fully then?

I’ve seen some folks—most notably, Cory D and Robb—doing some pretty cool shit stuff like dynamically pulling in content from online services like last.fm, Trakt, and Oku. The main problem with that for me is that I don’t use most of those services and I’m not looking to start now. (I try to be open-minded so if there’s something huge that I’m missing out on that’ll make my life easier then I’m open to learning about it.) The other problem is that I’m lazy and don’t want to build a whole bunch of integrations across multiple APIs. I am a bear of very little brain, so I like things to be simple.

My Requirements

This seems like a good place to talk about my requirements, which have evolved quite a bit over time.

Content Requirements

• Item title (text) and optional hyperlink
• Item category (Listening, Watching, Reading, or Tinkering)
• Choice of icon for each item - optional
• Episode name/number (so each new episode is not a separate item) - optional
• Progress (percent complete) to keep long-running items “current” - optional

Display Requirements

• Group items by category
• Display only “recent” items on my now page
• Display progress (if given) as a progress bar
• Display a default icon if no icon specified
• Display episode number/text as parenthesized text after the item title/hyperlink

Interface Requirements

• Must use a simple (i.e. easy-to-maintain) format/syntax
• All content managed in one place
• Ability to create, modify, and delete entries
• Ability to remove an attribute from an item
• Ability to refresh an item (update modified date) without changing item content

Generator Requirements

• Single source for all content - No APIs
• Leverage serverless technology - no dedicated infrastructure
• Minimal content validation - call out erroneous entries
• Keep content source file up to date - age off “old” entries
• Check for updates “frequently” throughout the day

The Architecture

The solution makes use of these main components:

• A YAML file, stored in a pastebin, for managing the content (That’s the part I interact with.)
• The Now Page generator code, which runs on AWS Lambda, Amazon’s serverless compute platform
• Data stored on the backend using DynamoDB, a NoSQL database
• An EventBridge Schedule to launch the Lambda Function multiple times throughout the day

The work flow goes like this:

1. User creates/udpates items in a YAML file.
2. The scheduler triggers the Lambda Function execution.
3. The Lambda Function loads content from the YAML file (via the omg.lol API), validates it, and stores/updates it in a DynamoDB table.
4. The Lambda Function updates the content in the YAML file, removing “old” items, while leaving the “recent” ones in place—so they can be modified later, if needed.
5. The now page content is generated from the update data and written via the omg.lol API.

Take a Peek

At the time of writing, these links contain the current code and content that I’m using to update my Now Page.

• Lambda Function source code
• YAML file (version 7)
• dynamodb_json.py - a Python library for converting content back and forth between ordinary JSON and the format expected by DynamoDB

Other Notes

• The PyYAML library is not available in Lambda by default, so I had to create a Lambda Invocation Layer to pull it in.
• I made a crude Mac/iOS Shortcut to build a formatted entry from a Share Sheet and copy it to the clipboard. It’s far from perfect, but it mostly works.