In my Atari ST video codec, the decoder maintains a codebook of small tiles out of which the video frames are composed. A certain number of tiles, here 32, are updated every frame, replacing less useful ones. This approach allows for high performance and decent compression.

Not sure if that can give you ideas: One of the things that really helped when we did video encoding was to reuse the content of the previous frame which often moves only by a few pixels or have only a couple pixels changing colors.

That’s a DOS-era codec, right? It’s way more advanced than mine.

Yes it is, but some of the concepts are probably possible to implement on a 8mhz 68000

With a full frame update there is no cycle budget left, but if whatever technique helps avoid full frame updates it becomes a tradeoff. Motion compensation for example might be interesting.

Hey, have you considered using whatever the blitter is doing to fold up a couple luts? You'd end up with floyd-steinberg style dithering which would look make the macroblocking otherwise invisible

@0xF33DF00D I’m doing dithering when converting the 256 color source image to 16 colors. I use 2x2 Bayer dithering for that. Bayer has the advantage that it looks the same for every block so there won’t be any artists from repeating at 2, 4 or 8 pixel intervals.

Ah, thats what I figured, probably would need a coprocessor to flip the pallet per frame (the Amiga did this?)

@0xF33DF00D Ah, I see what you mean – flip the palette per frame to get better color fidelity. That would work better if the framerate were 60Hz, but it is 15Hz.

I see, godspeed on your venture. 15 frames isnt a lot to play with even with old atari beam racing tricks.

Guess the tiles would work well with the Sega CD/megadrive...? Pity nowhere near enough RAM

Not sure about that. What you’re seeing here is about 56kB of codebook memory. Too much?

The Sega CD adds 255kb? And the main unit has 64, I think the VDP can map into either... Although I've not done anything serious on the Sega...

Could be interesting for the non new tiles to feed some amount of error of the prior frame forward so block choice tries for more energy preservation across time. But now sure how that works out without optical flow for the error.

That would probably create temporal flickering for blocks. What I’m doing is randomly updating 8 of the 32 tiles per frame. So with a certain probability, every block in the image reaches optimal representation at some time.

It might depend on how the error feedback falls off.

Definitely worth a try! I’ll report back.

@richgel999

how small is the file size? i presume the point is it will be super tiny and thus fast and tiny hardware can play it?

I usually get around 36kB/s, roughly a fourth of the original bitrate (which was probably optimized for 150kB/s singlespeed CD-ROM playback). Actually, I believe the ST(e) could just stream uncompressed video from disk. The key is to have compression without maxing out the CPU.

@chrono_000 Really cool! With 32 8x8 tiles with 4 bit colors + a 6bit tile index and a 40x20(?) tile map - again with 6 bit tile indices - that works out to 1648 byte per frame or ~40 kbyte/s at 24 fps. So on some frames you're updating less than 32 tiles? Or tile updates w. delta encoding?

@chrono_000 So what’s in a frame: - 32 new tiles (each 32B plus a 2B codebook index to update) - 40 columns, with each: - one 4B bitmask encoding which tiles follow (say N bits are 1, 0<=N<=25) - N 2B indices into codebook (0..2047) - 834B of audio samples

@chrono_000 Oh right, 2048 tiles would need 11bit - guess bit twiddling is too expensive/not worth it, hence 2byte? And audio is straight 8bit PCM? Would love to see an in-depth article about the whole thing at some point! <3

@chrono_000 Right, it’s a little wasteful but my main worry was getting the player tu run fast enough. Compression is a nice-to-have. Audio compression such as adpcm would max out an 8MHz CPU, so that’s off the table. I’m writing a short blogpost about this!

You know that feeling when you read over something again and again because it sounds like too much work 😭

damn son

I'd kind of forgotten how big a deal FMV was in the early 90s. This looks really great on hardware that wasn't really intended to do it!

The big missing piece before the mid-90s was the CD-ROM. Weird to think we went from 1.44MB to 650MB in a single step.

For a few years, CD-ROM was "more data than you could ever need"😁Certainly more than typical hard disks. I suppose it was really movies on disk that proved that wrong eventually.

Great solution!

Sweet !

Approved

Very interesting any link to how this technically works? I mean you got software to make the video right? Or build frames manually?

It's all here: vqatool is what transcodes VQA to STV videos. There is a file stvq.md in the vqatool folder if you're interested in encoding details.

that's cool as hell man