NEWS, EDITORIALS, REFERENCE
Another Progress Update
Some time has passed since my last post, near the middle of January. I like to keep up a pace of at least two posts a month. And I'm loath to slip behind in that goal. But, a rather nasty gastro bug that whipped through me, my wife and kids kept me out of action for well over a week. Combined with a particularly busy week at work this month, and time has been a bit short.
But there is also an exciting reason why the blog has been silent for a few weeks. I've put a lot of work into another major update to the Commodore 8-Bit Buyer's Guide. So I'd like to start with some updates about that. Then I'll move on to some updates about C64 OS, and even some fun news about C64 Luggable and the local Kingston, Ontario community.
Commodore 8-Bit Buyer's Guide
Let's start with why I decided to create the guide in the first place. When I came back to the Commodore scene after a hiatus of several years, my biggest surprise—and delight—was the discovery of how much activity there was. New games and new demos, yes, but also games publishing houses, fanzine magazines, and hardware. Lots of hardware.
I've always been a fan of hardware, ever since I was a Commodore-using kid. But, much of the new hardware was stuff I discovered in little dribbles. Every time I'd hear about something new, it blew my mind that it was out there and I didn't even know about it. Like, 1541 Ultimate was out there for a long time, and I thought SD2IEC was the only storage solution on the current market. And even SD2IEC came as a surprise to me when I learned about it. I didn't know about the multitudinous variations of form factor it was shipping in, and from all those different vendors.
When I learned about a new piece of hardware, but I didn't have time to look into it right away, I started a list. The list turned into a folder with photos, bookmarks, and links to documentation. Eventually, since I'd already created this website to serve as the blog for C64 OS development progress, I started putting pieces of cool hardware I wanted to recommend for C64 OS usage along the sidebar. But, this quickly got overwhelmed and when I revamped the site into its more-or-less current top tab navigation, I decided to devote an entire section just to hardware. And the Commodore 8-Bit Buyer's Guide was born.
I must admit, however, I had no idea at the time that I started it that it was going to grow to the size it has. Frankly, I had no idea that the quantity of products, projects, parts and kits available to buy was so high. Reflecting back on the idea to create the guide though, I am now convinced more than ever of its utility. The fact that I so dramatically underestimated how big the guide would become is the direct result of my ignorance of what was out there. It is safe to assume that many people, new, returning and even people who are active but focused on other things, are in a similar situation. It's just hard to hear about everything. The guide exists to help products get noticed. Even if you just look at the picture, and think, what the heck is that? You have a name and you have a category and often a sub-category. You can then look it up, or ask on Twitter. This is better than not knowing. Granted, I want to create more feature pages, but, it's a lot of work, and I've got a lot on my plate, so the feature pages are a bit slow in the making.
Some Stats About the Guide
It was last updated yesterday, February 21st 2019. At the time of publishing, the guide lists 352 products. Some of them are small, like replacement rubber feet for a C64c case. But others are big, like the MechBoard64, Ultimate64, TurboChameleon, and many others. Prices of products range from 65 cents to 259 Euros (388$ CAD!).
Prices are found in three different currencies, Dollars (USD, AUD and CAD), Euros and British Pounds. If you were to buy 1 copy of everything in the guide, you would have to spend:
- $4402.78 USD
- €3458.09 EUR
- £825.45 GBP
Or, if we convert that all, for comparison, you'd have to spend $9,386.10 USD. Needless to say, there is a lot of kit out there for us Commodore users to blow our hobby budget on.
The products come from at least 48 different suppliers. But, it's hard to count, because many people on Ebay and elsewhere resell the hardware created by individuals. And many individuals sell direct the one or two things that they produce. One thing I can say, it's a lot more activity than I remember in the early 2000s when it seemed like Creative Micro Designs was the last vendor standing, and they were on their way out. And it certainly isn't the case that new hardware of similar sophistication to the SuperCPU or CMD HD or CMD RamLink are not being created. The Ultimate64, 1541 Ultimate, TurboChameleon, and other products are plenty sophisticated.
What's New About the Guide Itself
I started with 12 categories, plus a "Features" section and a "Vendors" section, with a list of I think 10 vendors. I included my own Commodore Logo Patch in the featured section for people with a generous and helpful attitude to be able to help my work out financially and get a cool Patch in the process.
Some categories ended up with lots of products, and other categories ended up kind of anaemic. I had the further problem that some pretty crazy products out there seemed to defy basic categorization. How exactly does one categorize the 1541 Ultimate II+? Is it Audio? I mean, it simulates 2 SIDs, including extending them to 8 voices each, and remapping their addresses. That alone would constitute an exciting entry in Audio, alongside ARMSID and FPGASid. Not to mention that it also has the Ultimate Audio module and that audio-in port that firmware is yet to make use of. Or, should it be in Memory? It has a 16 megabyte REU, plus optionally a GEORAM cartridge, plus it's got extra features for backloading that giant REU, or to save an image of it to its storage. Speaking of storage, maybe it's just a storage device? What about that networking port though? Or is it a freezer cartridge, or an EasyFlash?
There was the opposite problem too. Where to put the C64 Reloaded. It's not a storage device, nor Memory, nor Audio, nor Video. I guess it's a component. But, it feels wrong to see an entire new C64 mainboard, the whole darn computer, stuffed into a category alongside a replacement LED, or a case sticker, or a single edge connector.
The result of all these considerations was to ditch the ill-conceived "Hacker" category. Distribute its listings between Storage, Hobbyist and Component. And to introduce a new category called simply, "Advanced", for mainboards, whole systems, and those rare but amazing products that span many categories.
Lastly, I spun my Commodore Logo Patch out of Features and into a new section, "Help Support Me". In various places throughout the site you'll find—what I hope is—a tactful little button like this one:
Want to support my hard work? Here's how!
The link from that pop over guides you towards the Help-Support-Me section of the Buyer's Guide. For the moment, that section is populated by the Commodore Logo Patch, but in the future I'll be posting a few other things: Versa64Cart's are on the way. Modified NES Controllers are a distinct possibility. And eventually the ability to purchase a boxed copy of C64 OS when it becomes available.
In this latest update, I have significantly changed the layout code. There is about 15% less markup, per item listed, and the CSS has been transitioned from quirky and unreliable floats to the much more modern CSS Grid. CSS Grid is a amazing. It simplifies the code while at the same time allowing for more sophisticated layouts, and overall better scalability down to mobile phones, although I'm still working on fixing a few issues at those small sizes.
A cool benefit of the switch to CSS Grid is the ability to change the layout "style" on the fly. Usually the layout is done to emphasize a product which, in my editorial opinion, is deserving of a bit more attention. One main product will be displayed larger, flanked by smaller related products. The user now has the ability to toggle the whole guide to a simplified grid layout. In grid layout, all products are displayed the same size, and fall into line in three neat columns.
Let's move along now to other topics.
C64 OS Updates
I showed off a build of C64 OS at World of Commodore in December, late last year. But, I have a confession to make. That build itself was from earlier in the fall, late summer. The reason is because I'd gone down the rabbit hole of terraforming a few major components of the core OS and there was no way I was going to be able to get things stable again in time for the show.
The good news is that I am done most of those important structural reforms and am now back in the seat of being very close to stable again, but with a number of important new technologies under-the-hood, some of which have gone through a couple of design iterations. So let's talk about some of these changes.
Splitscreen
I have recently acquired my first Amiga, an A1200. And I've been playing around with it to get a feel for how it works. It's got lots of nice little features, but in some ways it's simpler than I'd imagined. Of cource, simpler is a relative term. I've been biased by two decades of a modern OS like macOS (née Mac OS X). Some things in the Amiga are inspiring me, but one must bear in mind that a C64 is a significantly simpler machine, with far less memory and CPU resources.
I am absolutely satisfied with my decision to implment the graphical user interface of C64 OS in the C64's text mode, with a custom character set. It's much faster and uses much less memory, which enables things like realtime compositing, which I'll return to. But, there are those times when you want to see bitmapped graphics. Obviously. The trick is, of course, that a C64 is never locked in one video mode. I had always imagined, right from the beginning, that a Web Browser implemented primarily in text mode would allow you to select an image placeholder to download and view the image. The image data will be converted by proxy, so you're only ever downloading what a C64 is designed to render.1
The downloaded image data can go straight into memory reserved for bitmap data (behind the KERNAL ROM.) And C64 OS can provide controls for switching back and forth between viewing the image and viewing the text UI. That's the obvious move.
But, what would be even cooler is splitscreen mode. Originally I'd had the idea of dragging up the system status bar, from the bottom of the screen, revealing a graphics mode beneath, either Hi-Res or Multi-Color. When I dug into this I uncovered a whole new world of C64 tricks that I'd never dealt with before. Raster Interupts.
UPDATE: February 26, 2019
I went back through my Twitter timeline and found these tweets I made in October 2018, about my experiments with raster timing and splitting the screen between text and graphics. That was well before World of Commodore.
I think I just wrote my first routine that uses the raster interrupt to split the screen between text and bitmap. KERNAL and BASIC are active. A short basic program lets me move the split in 8px increments with +\- keys. #C64 I’m very excited! pic.twitter.com/gDZZkIfzVq
— Gregorio Naçu (@gregnacu) October 22, 2018
I’ve spent the night working on the splitscreen function in C64 OS. This is gonna be cool. #c64 pic.twitter.com/oFciEepjNZ
— Gregorio Naçu (@gregnacu) October 28, 2018
I'll go into more detail on this some time in the future. But here are the things I needed to deal with to make it happen:
- High Memory needs to be shared by graphix mode and C64 OS Utilities.
- The Mouse cursor needs to be consistently available across all three modes: Text, Graphix and Split.
- The split needs to be movable, both manually and programmatically.
- The graphix mode needs to support both hi-res and multi-color.
- The raster interrupt has to handle the split, and be stable.
- The interrupt handler has to be able to occur regardless of how KERNAL and I/O are mapped.
- Color data has to be moved into screen matrix memory when the split is moved up
- Screen UI memory has to be reconstructed when the split is moved down
- Mouse and Keyboard events have to be routed correctly when in Graphix or Split modes.
- Copying composited Screen UI memory to screen matrix memory has to pay attention to where the split is.
If you'll pardon the expression, that's a lot of shit to have to handle! And this is my absolute first ever raster interrupt programming. I'm no seasoned veteran. I'm reading manuals and VIC technical tutorials to figure out how this stuff works. So, it has been challenging to say the least. But I think it's going to pay off big.
In addition to just rendering an image, such as for an image gallery program, or a web browser that downloads an image, it's opened up new possibilities for user interface. I showed off the work I've done on a hand-pixeled UI for a program I'm working on called NES Tester. Here's what that looks like:
The NES Tester is a simple C64 OS application with two purposes. One is to test the 4-player adapter and input controller abstraction layer in C64 OS. You can use that with any joystick. And the other is to test my modified NES Controllers, in style. Including the select and start buttons.
The C64 OS application will have a text-based user interface, including instructions to drag up the status bar to use splitscreen. There it will reveal that this beautifully rendered NES controller has been loaded into graphics memory. The mouse is still active at this point, though, and you can click on the 4 port buttons to switch which port you're testing. You can also click on the switch button to toggle it to match how the physical switch is set. Then, pressing buttons on the controller should, if everything is working, highlight the depressed buttons in the UI.
It's a nice little example of how C64 OS's text-based UI is not as limiting as one might first worry about.
Context Drawing
I've updated the context drawing system, and moved it into the screen module. The C64's own KERNAL ROM has a primitive form of a "context drawing system." The KERNAL offers the routines PLOT and CHROUT. Plus, you can couple that with the PETSCII control codes for reverse on/off, text/graphics, color codes and a few others like clear and cursor controls.
To programmatically draw a user interface you can first call plot to set the start position, row and column, of the "context". You can then CHROUT a RVS code and a color code which sets those states into the screen editor. After this, you repeatedly call CHROUT passing it one PETSCII character at a time. The context determines how the character gets put on the screen. The PETSCII character is first converted to a screen code. Then, if reverse is on, it gets reversed. The screencode is then put into matrix memory at the cursor position, and the corresponding location in color memory is set to the current color. The cursor is then advanced to the right automatically.
This system is fairly simple. But, it is a form of context drawing, conceptually similar to what you find in, say, the HTML5 Canvas, which was based upon Quartz Graphics Contexts.
In C64 OS, a drawing context contains information about the memory buffers for text and color data, the width of the buffer, the width and height of a rectangle within the buffer and its internal scroll offsets. It also contains flags for color, reverse and for conversion from petscii to screencodes. Plus, it has one other flag, which is the direction of cursor movement, either horizontal or vertical.
I'll go into a lot more detail about this in a future post. But, in short, there are routines to backup and restore the drawing context, to change its properties and its constraint window. You then send it characters to draw, just as you would send characters with CHROUT. However, the C64 OS context drawing is much more sophisticated than the KERNAL's. C64 OS automatically clips drawing to the edges of the defined rectangle. And it does this while taking scroll offsets into account, all the while drawing into arbitrarily sized and placed memory buffers.
Lastly, there is a ctx2scr routine which is called to composite the contents of a drawing context at offset coordinates into the screen compositing buffer. This is what allows a window to be drawn to the screen but partially off screen and clipped to its edge. It seems like pretty basic stuff for a modern OS. Wowie, big whoop, you can drag a window partially off the edge of the screen. But, GEOS doesn't have movable panels of any description. Wheels, the big GEOS upgrade of the early 2000s, can't do it with the movable windows of the Dashboard, the movable windows in Contiki can't do it, and... this really surprised me, the Workbench 3.x windows on AmigaOS cannot be moved partially off the screen!
My earlier work on some C64 OS utilities and system drawing for the status bar and menu bar, etc. were done pretty much by hand. Now, they're transitioned over to the context drawing system which greatly decreases the amount of code needed to produce them.
Memory Management
A quick update on this. I published the plans in December for updates to the memory manager, in the C64 OS documentation memory management article, Dynamic Memory Management and Block Structure.
This has been implemented, tested and debugged. These dynamic memory management upgrades, combined with the finalized context drawing system, lay the foundation for the toolkit.
Toolkit
Way back before I was ready, I created a first version of the toolkit. But it quickly became apparent to me that it wasn't being implemented in a scalable way. So I have completely redesigned how it works. I gutted 90% of the original code, completely redesigned the object hierarchy, and reimagined how even to implement a system of object oriented classes.
I'm now in the process of rebuilding the fundamental UI classes. But in a much more sustainable way. They backend on the dynamic memory management, allowing one class to internally instantiate other classes. And they use the context drawing system as the foundation for each class to draw itself. A class that supports children can push its own draw context to the stack, then modify the context by shrinking and insetting it, and then ask its children to draw themselves. They then are constrained to draw within that new context. And so on recursively.
A lot more to talk about on this front in a future post.
Loader
At first, if I felt like the system needed some functionality, I decided which module of the OS it most reasonably fit into and I added it in. Over time, though, the system code has grown. And it will continue to grow as I flesh out the Toolkit. It occurred to me though that not all the code needs to be in memory all the time.
For example, there are routines designed to load in the human-editable menu file from an application bundle, parse the file into in-memory structures, and then recursively link all those structures together so they can be rendered by the menu system. There are also the corresponding routines used to deconstruct the current menu structures and free the memory when an application is quit. What occured to me is that this code is only useful during the transition from one application to another.
After an application is quit, its menus need to be deconstructed. Then, the next application's menus need to be read in and constructed. Once the memory structures are linked together, the code that does all the parsing, allocating and linking is just deadweight until the next application has to be loaded in. So, I moved all of this code to a temporary program called the loader. When a homebase app is used to launch an app, it constructs the incoming app file reference, and loads and runs the loader. The loader does a bunch of intermediate work to clear out the homebase app, and then usher in the app being loaded. The binary of the app being loaded overwrites the loader itself, such that none of that code is left in memory while an app is running.
I've done something similar with the booter. Many elements of the OS, such as drive detection, realtime clock access, initializing the memory allocation table, loading the character sets, etc, is done by the booter. But once in place, the time is set, the detected drives table and the memory allocation table are left in memory, but all the code that constructed them in the first place is not.
That's it for C64 OS updates. And sets the stage for many more technical deep dives in the future.
Local Retro Computing News
That's enough of the technical stuff for now. In other news, Jérémie Marsin (of @DSidedGames) has stepped up and—with some help and encouragement from yours truly—has organized the Kingston Retro Computing Club. The beginnings of a website for it can be found here: kingstonretrocomputing.club. Great domain!
We're meeting once a month, near the end of every month, at Queen's University. The exact room is still a bit in flux. Our second meeting is not in the same room as our first meeting. But, I'm sure that will get sorted out and we'll settle into a groove.
We had only four attendees, but, the weather was a terrible Canadian winter night. Rob, Brad, Jérémie and I hung out, watched some demos, played some classic games, nerded out over hardware we'd never seen before and had a great time. If you're in the area, check the schedule and make an appearance!
In the interim I've installed 5 front push buttons on C64 Luggable, for controlling the 3 buttons of the 1541 Ultimate II+, and the next and previous buttons of the SD2IEC. I also got some heat sinks to install on the VIC and SID chips, but I didn't have time this month to get them properly installed.
I've been a Commodore user for 32 years. And in all of that time, I have never been to a local computer club where I could bring my C64 and have other people take an interest in it, until now. It's a new chapter in my Commodore life.
- Which you can read about here: http://services.c64os.com/about#image. A suite of online services will be critical to getting C64 OS to have independence on the internet. [↩]