WoC19 Video playback with 1541 Ultimate II+ (2/2)

This post is part 2 of a two-part follow up to my review of World of Commodore 2019. If you're interested and you haven't yet read part 1, WoC19 Introduction to C64 OS, please check out that post.

I had a one hour block of time to present at World of Commodore, but I asked them to split it into two half-hour blocks, because I wanted to give a talk about Video Playback on the C64 with 1541 Ultimate II+. It doesn't really need an introduction, because, that is the whole point of the presentation itself, which we'll get to shortly.

Additionally, I mentioned in part 1 that my presentations were given with slide presentations from my C64 Luggable. It is hard to tell in the YouTube videos exactly what is going on because the camera spends most of its time focused on the screen and only a few shots actually include me. And of those, you don't see much of my C64 and probably don't notice what's going on in my hands. However, if you check the thumbnail image for this post, above right, you'll see that I'm holding a NES controller.

I do love NES controllers for joysticks on my C64, as I've talked about here and here. But it turns out that a NES controller also makes for a really great, inconspicuous, one-handed slide clicker. The two red fire buttons make for easy back and forward controls that can be triggered with your thumb by holding the controller such that the d-pad is buried in your palm. Additionally, the long 6' cable provided me ample freedom of movement.

So, before we get into the presentation I wanted give a technical overview of how the presentation software works.

(If you really couldn't care less about the presentation software, you can skip it.)

Slide Presenter for your C64

I've always liked the idea of using my C64 for more things. I don't love the idea of going to a retro computer show, and then pulling out a Mac and using it to present some fancy slide show. I know that that could be useful, but it also feels bittersweet.

Here, let me use this other computer platform that beat Commodore in the Computer Wars to show you this thing I want to talk about on my C64. Sad Face :(

I didn't know how quite to go about using a C64 to present. Not until I saw this recent video about AcheronVM from VCFMW 2019. He just unapologetically uses the C64 to display plain text, lightly formatted with spacing and asterisks and the occasional PETSCII underline, to convey the essential information. As it turns out, that's actually not that bad. The C64's relatively low resolution of 40 x 25 characters means that on the big screen those words end up being a decent size to read, and also forces you to make your slides in point form.

You can checkout or download the source code to my Presenter program on GitHub: Presenter sourcecode.

That source is in 6502 assembly, for the C64, in TurboMacroPro format. Absent from the source are the include files that define some of the constants used for KERNAL calls and I/O chips. It also omits the include file for the BASIC prelude that bootstraps the ASM program. The only part that is not published elsewhere and widely known is in the pointers.s include. Pointers.s is a standard part of C64 OS, but this Presenter program doesn't require C64 OS, it's just a regular C64 program.

Pointers.s defines some macros, only three of which are used in the Presenter.

rdxy  .macro
  ldx \1
  ldy \1+1
.endm

ldxy  .macro
  ldx #<\1
  ldy #>\1
.endm

stxy  .macro
  stx \1
  sty \1+1
.endm

Very simple. Each macro just takes a 16-bit word, and uses X for the low byte and Y for the high byte. rdxy reads a 16-bit value stored at the provided address. ldxy loads a static 16-bit number into X and Y. And stxy writes the contents of X and Y to some address, X as the low byte, Y the high byte.

How it works

The program is very short, less than 512 bytes. The name of a slideshow data file is hardcoded at line 290. You have two options, you can change this line and reassemble the program once for each slideshow you have. Or, you can set this line to always search for a generic filename, like "slideshow.pet", and then copy-replace the slideshow file you want to open to slideshow.pet before running the presenter. What I did for my two presentations was just assemble the presenter twice.

I'm absolutely loath to write a more sophisticated means of picking a file to open, because this will eventually get ported to C64 OS, and in large part the point of the OS is so that you don't have to (re)write a file picker for every application! So, this is a good example of how C64 OS would be useful. Just use the Files utility to pick the file to open. Okay, that's for the future. For now, the filename to open is hardcoded.

It starts by wedging a scanbut (button scan) routine into the IRQ handler. Then it clears the screen, switches to uppercase/lowercase mode, sets the border and background to white, and the text color to dark grey. These colors could obviously be customized each time the Presenter is assembled. But a logical extension to the data file format would be to add codes to support changing these colors during the runtime of the presentation.

Next, it initializes a "Slide Pointer" stack. This stack supports up to 255 slides. If you think you need a slideshow with more than 255 slides, you are a crazy person, and you'll probably run out of main RAM first anyway. We'll return to the slide pointer stack in a minute. It then loads the slideshow data file into memory, following the Presenter binary, from $1000 up.

Now that all the data is in memory, the rest of the program goes into a loop, processing the data and outputting it to the screen. Special formatting codes are found within the file that indicate when a new slide begins and when to pause the output. At every pause, a sub-loop is entered that checks for user input to determine where to navigate to next. Eventually the slide data must include an end code marker, which tells the Presenter that the last slide has been reached and will quit the user back to BASIC when the user chooses to navigate past the end marker.

The Presenter Data File Format

The slideshow data format is plain PETSCII text. The text is preformatted. It is recommended to layout the slides in a standard 40-column text editor, such as Novatext (my personal favorite, it comes bundled with Novaterm.) This allows you to see how your content will fit on the screen. Spaces and carriage returns are output as they are encountered.

PETSCII control codes are supported, naturally, because everything is being output via the KERNAL's CHROUT routine. Therefore, if your text editor supports it (for Novatext this is sketchy), you can put the PETSCII codes to change text color, or turn reverse on or off, inline with the content. This will unfortunately detract from the WYSIWYG preformatting of the text. I didn't use any of these options in my own presentations, but you certainly could do it.

There are 3 special control codes. "!", ",", and "." (exclamation point, comma and period.) These characters are only interpreted as control codes if they immediately follow a carriage return. Aka, if they are the first character of a line. The control characters themselves are not output to the screen, so they will have an effect on the WYSIWYG layout, but in practice it is very slight.

The exclamation point defines the start of a new slide. When this code is encountered, the output is automatically paused, waiting for user input from the joystick/NES controller. If the user chooses to proceed as normal, the address of this new slide marker is pushed to the slide pointer stack. The screen is cleared, and the following data starts being output to the screen up until the next control code.

This allows you to define slides. Just put a ! (bang) between two lines, and the output will pause there, wait for the button to be pressed, clear the screen and output up to the next new slide marker. That's the simplest way to make slides, and it works.

In practice we usually want the contents of a slide to be progressively revealed. For this we use the comma, which is an analog for pause. When the pause marker is encountered, it works just like the new slide marker, it pauses and awaits user input. But when the user chooses to advance, the screen is not cleared. Thus, you can put pause markers at the start of several lines and each line will be output only following the press of the next button.

The period is used to mean stop. When the stop is encountered, output is paused, but if the user presses the button to advance, the program will cleanly quit back to BASIC.

Navigating the Slideshow

Each time any of the three control characters is encountered, the output is paused until the user provides input. On the joystick the standard fire button advances. Pressing Up, (which on the modified NES controllers is the second fire button) causes the presentation to return to the top of the current slide.

To return to the top of the slide, the screen is cleared, the data positional cursor is reset to the address last pushed to the slide pointer stack, and output continues from there until it hits the next control character. This is useful if you happen to go too far, maybe you've hit fire three times this slide, but you wanted to only step into the slide twice. You can hit Up, and then hit fire a couple of times to step into the current slide again.

At times you may accidentally step past the end of a slide, and into a new slide. Use the left button on the joystick (or d-pad) to go back to the top of the previous slide. It accomplishes this by first popping one address off the slide pointer stack and then executing the top-of-slide routine. It is intentionally harder to trigger this feature, so you won't do it by accident. But, if you want to, it's a quick way of hopping back a slide at a time all the way back to the start of the presentation.

And that's it! That's all there is to it. But it served me quite well. And you are free to use the code as you see fit, it's open sourced, on GitHub.

And now on to my second presentation at World of Commodore.

Intro to Presentation

For this presentation, I was already setup from the previous. I was also quite a bit more relaxed. I was quite nervous that something in the C64 OS demo would go catastrophically wrong, crash, lockup, or otherwise embarass me. For this presentation, it was just a lot more low-key, and I was able to have a bit more fun.

The previous year, at World of Commodore 2018, I had an early demo of some videos playing on my C64 Luggable. But they weren't really ready to show off. They required a fair bit of manual setup. This year, I'd added a new video mode, improved the video creation tools, improved how the player works, and so I thought it was ready to show off.

Here's the transcript of my talk, which I deviated from only very slightly.

World of Commodore 2019 Presentation

Video Playback with 1541 Ultimate II+

An audio version of this talk is available in a Hacker Public Radio episode, here.

Hello, welcome back to those who just attended the Introduction to C64 OS.

My name is Greg Nacu, I live in Kingston Ontario, and I've been a Commodore 8-Bit user for as long as I can remember. I've recently come back to the Commodore in the last few years. Today I wanted to talk about Motion Video playback on the Commodore 64.

Ever since I was a kid, I wanted to have full motion video on my C64. The first computer I saw video on was a 486 PC running Windows 3.1. It was about a 3 second clip, the size of a postage stamp, with no audio. It was a clip of the Enterprise from Star Trek The Next Generation jumping to warp.

I was amazed.

But I wanted it on my C64.

On the C64, there are two native bitmap modes, hires and multicolor, and the color and bitmap data come in two or more different regions of memory.

For the fullscreen, bitmap data is 8000 bytes, just under 8K.
Color data for hires mode is 1000 bytes.
Color data for multicolor mode is 2001 bytes.

If you're curious, the extra 1 byte is the common background color for the whole screen.

Now, if we make the bitmaps black and white, and perhaps dither them for tone, then color data can be ignored. We can just preload black and white as the background and foreground colors, say, for every cell on the screen in hires mode. If we did this the minimum memory size of a fullscreen frame is just under 8K.

There are several challenges with getting video to run on a Commodore 64.

The first problem is memory size.

But the C64 only has only 64K of memory. So, in theory, there is only enough memory to store 8 frames. (64 / 8 = 8 frames.) In practice there is even less because you need some space for I/O addressing, and you need some space for your code. So maybe we could fit 6 frames.

How do we deal with this? Well, one approach, we could stream the video into memory, from storage.

The problem with that is, over the IEC serial bus at least, we can only read about 6K per second. According to the JiffyDOS user manual. That's not enough for even 1 frame per second. And pulling that off completely taps out the CPU and the serial bus, so you've got no resources left for audio of any kind.

Okay, well, we could augment this approach. You can compress the frames.

The problem with compression is that you need CPU time to decompress, and any CPU time spent decompressing data is time that can't be spent transferring data over the serial bus. But there might be some gains to be made there. Some clever work might be able to pull 2 frames per second or something, still with no audio.

To really do video, what you need is memory. More memory and ideally more CPU horse power. It would seem as though the Commodore 64 is hopelessly unsuitable to playback video.

Skip a few years, into the mid-to-late-90s and we got the SuperCPU from Creative Micro Designs. The SuperCPU was an accelerator for the C64 and C128, that ran at 20Mhz, had a 16-bit mode, and could support up to 16 megabytes of RAM, with an add-on SuperRAM card. NOW we're talking, right?

In the early 2000s at the Commodore Expo in Chicago I demoed a video player that I'd put together. Here's how that worked.

Jolz Maginnis from Australia had written a tiny Unix-like multi-tasking operating system for the SuperCPU called WiNGs, and it included the beginnings of a graphical user interface in hires mode. He had also included a command line JPEG renderer that would convert JPEG images into dithered 4-color hires bitmaps. I thought, hey, if I could rip frames from a video and scale them down to a series of small JPEGs, I could write a program for WiNGs to pre-convert the images to C64 format. Then pack them into one file, combine with a raw uncompressed PCM audio segment, and holy crap this might work!

And it did work. It really worked. I dazzled people at the show with video, with synchronized digital audio.

That was the power of a SuperCPU.

To create a video, I used QuickTime on a Mac to export the frames to JPEGs, and then a script to scale the images down. Let's say it was 2 kilobytes per image, a thousand frames weighed in at around 2 megabytes, which had to be transferred to my C64. The fastest way to do this, at the time, believe it or not, was to zip them up and transfer to an IDE64 harddrive over a 56K modem.

In case you forget what this was like, it took forever.

But, then I had to unzip a multi-megabyte file on a C64. This also took forever. But then, I had to run my movie conversion tools, to decode and convert a thousand JPEG images. This process literally took several hours to complete, even at 20Mhz. But it worked. The audio was ripped and decompressed and downsampled also using QuickTime. But it too had to be transferred to the C64 over a modem, where it could be combined with the video. A final video file would max out around 15 megabytes for just over 2 minutes of playback.

To playback one of those videos, you first have to load the file into memory. Using an IDE64, which is probably still the fastest native mass storage solution for the C64, it took well over a minute to load all that data into the SuperCPU's memory.

From there, memory contained an audio segment, and an uncompressed video frames segment.

The audio was played back with a DigiMax. DigiMax is a stereo 8-bit digital audio converter that plugs into the user port. These are still commercially available, by the way. DigiMax was created by my friend Vanessa Dannenberg of Digital Audio Concepts and is currently sold by Shareware Plus.

When playing back video, it is most important that the audio not skip or stutter. For some reason the brain can ignore minor interruptions in what you see, and it's not that annoying. But if the audio skips or cuts out, even for a moment, it's extremely annoying. So to keep them in sync, the video frame was changed after every so many bytes of audio, and occasionally a frame might have to be dropped to keep them in sync, but this was not noticeable.

Now, bear in mind, WiNGs provided a multi-tasking and multi-threaded KERNAL.

As bytes were being sent to the DigiMax with one thread, the code would trigger another thread to transfer the next frame of video from SuperCPU memory to a view buffer. The view buffers were then composited together by the OS and transferred to main memory to be displayed by the VIC. 20Mhz is pretty fast, so there was plenty of time to do all of this. In fact, the SuperCPU was so fast, that it was possible to play up to 3 videos at the same time! Each in a separate window.

Amazing.

As a demo, the WiNGs movie player was cool, but it had serious drawbacks.

Creating videos was so laborious, I only ever made about 4 or 5 samples. And I only know of one other guy who also went through the painstaking process of creating a video. It was a clip of himself at some Euro C64 party.

The second problem is that, sadly, the SuperCPU is no longer in production, and it is now considered very rare. They are very expensive when they come up on Ebay. Also, many C64 diehards have balked at the accelerator, saying that the SuperCPU-equiped C64 is not really a C64. And they have a point. It's more like the Amiga Vampire accelerators, that take over the entire machine.

And lastly, even if you had a video pre-converted, and the SuperCPU to play it back, with the SuperRAM card to hold it in memory, and an IDE64 for maximum load speed and a Digimax for audio, you had to wait almost as long to load the video as the video itself took to playback.

It's cool, but, the drawbacks are serious. They will preclude most people ever actually seeing it work on their very own C64.

Skip ahead another 15 years or so, and here we are.

The SuperCPU is no longer available, and all the limitations on memory are still the same. But, there's a new piece of hardware in town. The 1541 Ultimate II. I should add, anything the 1541 Ultimate II can do, can also be done by the Ultimate 64.

I have an epic indepth review of the 1541 Ultimate II+ on my blog, at c64os.com. If you want to know more about what all it can do, I recommend reading that post. But here are the important details for today. It plugs into the expansion port. It can be configured as a standard Commodore REU but with a capacity of up to 16 megabytes. It allows the C64 to access USB storage devices. Plus, it has something cool called the Ultimate Audio Module. And best of all, the 1541 Ultimate II is a commercial product that you can buy today, that is actively supported and continues to be updated with new versions and features.

When I learned about the 1541 Ultimate, and I bought one for myself, my brain lit up again for how to get video playback on my beloved C64. But this time, advanced tools on the Mac are much more readily available.

So, here's what I do to create, what I'm calling an Ultimate video file:

Find a cool video on youtube that's under 2 minutes and 30 seconds long. There are zillions of these, but my favorites are movie trailers. I take the address of that video, and paste it into a command line PHP script, on a Mac, and less than 5 minutes later, I have an Ultimate Video file, ready to be sent to the C64 for playback.

The script uses several command line tools. YouTube-dl downloads the video from the YouTube URL to a local file. FFmpeg is used to extract and downsample the audio to mono 8-bit raw PCM. FFmpeg is used once more to extract the video frames to a series of PNG image files. The framerate is configurable, with options of 6, 8, 10, 12, 15 and 20 frames per second. 10 is what I usually use. Lower framerates can be used to get longer playback. And for shorter videos higher framerates can be used for smoother playback. The frames are then scaled and converted to C64 image format.

Last year I had black and white dithered working, with a converter to C64 hires format that I wrote. Now, there is an option to use the C64 Graphics Library, by Pasi Albert Ojala (sorry about my pronunciation), to convert the frames to dithered multicolor format. The script then packs the frames together with the audio, and the result is a file format that is similar to the WiNGs movie file format.

Ultimate Videos, on a modern Mac or PC, can be created in a matter of a few minutes, rather than hours and hours on a SuperCPU.

The options for transferring large files to a C64 have also opened up. With SD2IEC and the 1541 Ultimate, we can now access SD cards and USB sticks formatted for PCs directly on our C64s. So you can pop the video file onto a USB key and stick it in your 1541 Ultimate and the file is "transferred to your C64" in mere seconds, rather than hours over a dialup modem.

Now, how about loading the content into expansion RAM? Think about this, the SuperCPU could have 16 Megabytes of memory, but it wasn't a storage device. It's only source of content was the C64's own data bus. And so the C64 itself had to be used to read data from a storage device and copy the data into the SuperCPU's memory.

The 1541 Ultimate is different. It's an REU, and a storage device, and more, all rolled into one. The C64 can command the 1541 Ultimate to load a file from USB storage directly into the REU. This data transfer does not go across the C64's own data bus. Which allows the transfer to happen very quickly. The entire REU can be filled, all 16 megabytes, in just 4 or 5 seconds. But once the data is in the REU, now the C64 can access it the same way, and with the same speed, that it can access any content in an REU.

Next we have audio. A SuperCPU is not a digital audio playback device, it's just an accelerator. The player program had to read bytes of audio from SuperRAM and push them to the DigiMax on the user port. Just pushing digital audio on a stock C64 is CPU intensive. Let's say the audio is sampled at 11Khz, with a single mono channel of 8-bit samples. That's over 11000 bytes a second. You only have a million cycles per second, which leaves you with 90 cycles per sample. And it takes about 20 cycles or so, on average, to copy a sample from memory out to the user port. So it takes around 20% of the CPU time just to push the raw audio data. If you upped the audio to 22Khz in stereo, it would take 80% of the CPU time just to push that much raw data out the user port.

The 1541 Ultimate has another trick up its sleeve. The Ultimate Audio Module, which I mentioned earlier. When enabled, the audio module occupies a few bytes in I/O space. The C64 can issue commands to it, to play back samples of audio stored in the REU. You specify the sample bit width (8 or 16 bits) whether its mono or stereo, the sample rate, the start address within the REU and the length of the data. And it plays the sample.

But here's what's even better. The ultimate audio module plays the samples directly from the REU, without the CPU needing to push every individual byte. So you can trigger the audio playing, and have 100% of the CPU left over to do whatever you want. Very cool stuff.

So the way the Ultimate Video player works is, we load the movie file into the REU, then the player tells the Ultimate Audio Module to start playing back the audio region of memory, and we connect the frame changes to the raster IRQ. The framerate of playback is timed by dividing the IRQ framerate. Every so many VIC-II refreshes, we tell the REU to copy the next frame from REU memory into main memory for the VIC-II to render.

And, bingo, we've got a full motion video playing, synchronized with digital audio.

Okay, so now I'll give you a little demonstration.

There is actually a history to the choice of that video. The Incredibles 2 came out in 2018, and I converted that video sometime last year. But the original The Incredibles movie came out in 2004. The trailer for the original is one of the videos I converted for the WiNGs movie player, and showed off in Chicago 15 years ago.

That was an example of a hires, dithered black and white sample.

And here's another example:

Isn't that great? Okay, one more.

I've been working on building a suite of webservices that convert and simplify web content for immediate consumption by the C64. There is already a library of relocated small SIDs, an image conversion service, and an ATOM RSS feed interpreter. I have others that are underdevelopment.

The goal is to make the video conversion scripts into a webservice. The idea is that you'll be able to call the service, send in the URL to a YouTube video, plus some parameters such as color or black and white video, framerate, audio quality, and perhaps start and end time stamps. The conversion will happen on request and the C64 formatted video file will be returned as a download.

And that's it! Does anyone have any questions?

Meta Analysis

I was much more relaxed giving this presentation. I didn't feel as pressured for time, and it was fun.

The only thing that made me feel a bit dumb is that there are other people out there doing amazing work with Video Playback on the C64. The total amount of work I had to do to get these working was actually quite low. There are no special video tricks, or fancy video modes.

I realize that the quality is not as good as the C64 is capable of producing. However, they are very easy to create. They can theoretically be produced by a webservice (mentioned in the transcript above, but I forgot to say that part during the actual presentation). They work perfectly on both PAL and NTSC machines. But, additionally, because the requirements on the C64 are so low, you have to imagine that my plan is to get these videos working inside the context of C64 OS.

How would that work? Specify to the OS where the graphics buffer should be, and the format (hires or multi-color). This allows you to drag the status bar up to set the split position. And as the IRQ frame updater executes, it checks the split position, and has the REU only copy a segment of the bitmap into main memory.

In other words, the idea is to get the video playing in splitscreen, leaving the mouse active and with the ability to interact with the menus and UI above the split at the same time. So, the quality is lower, yes, but the idea is to make them much more interactable, like in a modern video player. Play/pause at least, but also options to toggle repeat, maybe the ability to skip forwards or backwards or to a specific timecode, and possibly the ability to create a list to queue up videos to play one after the next.

There are a lot of cool options.

One last note. I was contacted by someone who was at the show who had some ideas for improving the quality by pre-processing the frames. I admit, all I'm doing is running the PNG image data through the C64 graphics library to convert to KOALA format, multi-color with some light dithering. There are undoubtedly some improvements that could be made, such as applying brightness and contrast changes, or swapping in an image converter that just does a better job.

I will try to get this converter working on the machine that hosts services.c64os.com, so that others can try it out. I'll write another blog post about this when it's ready to use.

Do you like what you see?

You've just read one of my high-quality, long-form, weblog posts, for free! First, thank you for your interest, it makes producing this content feel worthwhile. I love to hear your input and feedback in the forums below. And I do my best to answer every question.

I'm creating C64 OS and documenting my progress along the way, to give something to you and contribute to the Commodore community. Please consider purchasing one of the items I am currently offering or making a small donation, to help me continue to bring you updates, in-depth technical discussions and programming reference. Your generous support is greatly appreciated.

Greg Naçu — C64OS.com

Want to support my work?

C64 OS version 1.04 is now available

The best way to help C64OS.com grow and continue is to purchase a commercial copy of C64 OS version 1.04.

Join the growing C64 OS user community. Take advantage of the powerful and useful features available now, and get access to run new and future software made for C64 OS.

Learn More Order Now

C64OS.com is hosted on a Digital Ocean droplet.

Digital Ocean is a cloud hosting platform, built for developers. They make it simple to get started. They have intuitive administrative tools and reasonable prices. And they can scale as your needs grow.

If you sign up with this referral link, you'll get a $100 credit, and after you've spent $25, I'll get $25. If you are looking for a cloud platform to host your website or web app, this is win-win.

Sign up for Digital Ocean with this refereral link.
Save money and help C64OS.com at the same time.

No, thank you.

Want to support my hard work? Here's how!