Reconstructing game footage from a Game Boy's memory bus (37c3)

How do you capture a video from an 1989's Game Boy without modding the original hardware? With an adapter cartridge that spies on the memory bus! Let's talk about how to reconstruct the Game Boy's memory state, emulate its graphics unit and then encode the image into an MJPEG stream for anyone to use as a USB video class device. In realtime. On an rp2040 microcontroller. The original goal of the open source project "GB Interceptor" was to capture gameplay for one specific game: Tetris. In order to live stream a Tetris tournaments from the contestant's personal Game Boys, the idea was to create an adapter that goes between the Game Boy and the game module to analyze the communication on the memory bus and reconstruct the game state. It turns out that it is actually possible to reconstruct the entire memory state of almost any game and in fact create an rp2040-based adapter that acts as a USB video class device offering the on-screen game footage in realtime. Players can simply put this adapter into their Game Boy and use it like a webcam without additional drivers or knowledge. An essential aspect of this concept is that the Game Boy basically runs all of its code directly from the ROM module, which makes it possible to directly follow the program counter of its 8bit CPU regardless of how the code branches. An image can then be recreated by emulating the graphics unit (PPU). However, there are many edge cases like interrupts, data from registers that are not visible on the bus, the link cable, DMA operations, synchronization of CPU and PPU, game bugs and even bugs in the Game Boy hardware itself. In this talk I will show how all this is done just on an rp2040 with spare cycles to encode everything as a 60fps MJPEG stream. I will shine a light on the edge cases - those that were solved and those that might just be unsolvable with this approach. And I will take you on a sightseeing tour through the 8bit hell that drives our iconic handheld from 1989. about this event: https://events.ccc.de/congress/2023/hub/event/reconstructing_game_footage_from_a_game_boy_s_memory_bus/