CS and AI
6 posts
Every few months I end up with two copies of the same photo library on two different drives, and I don’t know which files actually differ between them. File Explorer won’t answer that question, and I usually need the answer before I delete one of the copies.
Every coding agent I use keeps its memory in a local folder — .claude/, .cursor/, .agent/. That works until you switch machines, and then the agent has amnesia. The context it built up about your project stays on the laptop where it happened.
Reacher is the environment I had the easiest time with, and the one that taught me the most — because it contradicted what I’d learned on Tennis a year earlier.
If you’ve ever had a folder of papers you meant to read, you know the problem this came out of. I wanted something that would take a PDF and hand me twenty sentences worth reading, so I could decide whether the paper deserved an hour.
After the Banana Collector I moved on to Unity’s Tennis environment, which is harder in two ways. The actions are continuous, so DQN’s trick of taking the max over four actions doesn’t work anymore. And there are two agents, each one sitting inside the other’s environment.
This was the first RL agent I trained end to end. The environment is Unity’s Banana Collector: a square arena full of yellow and blue bananas. Yellow is worth +1, blue is worth −1, and the agent has to work out which is which from a stream of numbers.