Supporting IDE64

Welcome back one and all. I hope things are going well for everyone. This post is going to be a shorter talk about IDE64 and what it took to get it supported by C64 OS. I've classified this as a software post, and then I've gone ahead and made the icon a picture of a piece of hardware. At some point the hardware and how it works becomes a software issue. I'm going to talk briefly about the IDE64 and what I've learned about how it works, before getting into the changes I made in C64 OS.

Before we get started I'd like to thank everyone who purchased a Versa64Cart that I had available on this site. Your support is very much appreciated. I have now sold out of the stock that I brought in and put together as part of my experiment in how one can go from online open-source hardware to actually having the product in your hands. I wrote about the process in the post Versa64Cart and C64 ROMs. I learned a lot and some of the things I learned are even relevant for this post and how IDE64 works. It was also a lot of fun, and I got a Versa64Cart into the hands of a bunch of people who wanted one.

I still have Commodore Logo Patches available, if you're looking to support my work here. And I have some work to do on updating the manual and assembly instructions for the Promenade C1 clone, the Promenade Model D. But soon I'll have those available here as kits for purchase.

Now let's talk IDE64!

My brief history with IDE64

Back in the late 90s I ordered a version 2 IDE64 from the Czech Republic. It was a bit of a novelty in those days to order hardware from the middle of Europe, and I didn't know anyone else who had an IDE64. But I got one, and I fell in love with it. It was unbelievably fast, and supported large partitions on cheap IDE Harddrives. And as a bonus you could hook up a CD ROM and read CDs that were formated for PCs, right from the READY prompt.

Shortly thereafter I started helping Jolz to debug some drivers and components of WiNGs, and the IDE64 absolutely flew with WiNGs (pun intended). With a shell and a GUI and 16MB of RAM at 20Mhz on a SuperCPU, and a TCP/IP stack, it really made the C64 feel practically modern, albeit with C64-era graphics. The only problem was that so few people here in North America had an IDE64. This seemed a real shame considering how fast and amazing the device was.

Around that time the IDE64 guys released version 3 of the hardware which included a Compact Flash card slot at the end. This was even better, because CF cards were cheap and abundant and it was far smaller and you didn't need the ribbon cable and bulky harddrive with the external PSU to spin it up. I was frequenting tradeshows, like the SWRAP Expo (Chicago) and the LUCKI Expo (Louisville) and of course World of Commodore in Toronto. So I decided to do something a bit daring, and purchased 10 IDE64 v3s. I was only like 19, I didn't have a job, but in the year 2000 I dropped a $1000+ on a box of C64 hardware I intended to resell. I took them to the next show, each packaged in a vacuum-sealed plastic bag, along with a color printed manual, a battery, an IDE64 Utilities disk (1541) and a 1581 disk distro of WiNGs. They sold like hotcakes, and I didn't bring any of them home with me.

I did that a couple more times, I think I sold around 30 in total. Eventually the IDE64 v4 was released. But by that point, I was nearing an abyss. Jolz had stopped developing WiNGs. The IDE64 v4 wasn't properly supported by the WiNGs driver, and I had no clue how to fix it. The SuperCPU was the major bottleneck though. It was no longer being produced, and my happy dreams were crumbling.

Jump forward a decade. IDE64 is still commercially available, which is fantastic. But also, the world is much more comfortable ordering hardware on the internet from distant places. The IDE64 is now listed in the Commodore 8-Bit Buyer's Guide, but I no longer feel the need to import and resell them. Now, I've been working on C64 OS, which does not require the SuperCPU, and is able to run on any breadbin C64. But I knew from the beginning that the IDE64 must be a supported storage device.

IDE64 and how it works

C64 OS requires a storage device that supports subdirectories; native subdirectories that are exposed to the C64's READY prompt that, with standard DOS commands, can be navigated, created, removed, and used to organize the hierarchy of C64 OS system and application files. The CMD HD, FD, and RamLink are the grand-daddies of Commodore 8-bit storage devices with first-class support for partitions and subdirectories.¹ The IDE64 came next, and also supports both partitions and subdirectories with a DOS command structure that is very compatible with CMD's devices. And later still came SD2IEC that, as I've gone on about at length before is like a teeny tiny CMD HD for 1/10th the price.

There is a major difference between IDE64 and either an SD2IEC or a CMD HD though. SD2IEC and CMD HD are just regular IEC bus storage devices. There is nothing at all magical or mysterious about how they work. The C64's original KERNAL ROM has all of the IEC bus protocols built in for communicating with disk drives. And Commodore 8-bit computers have never had the DOS (the disk operating system) built into themselves, but rather the DOS has always been built into the storage device. I wrote at some length about this in the post KERNAL, File Refs and Services.

It was therefore easy and a natural progression for, say, the CMD HD to be able to add support for partitions and subdirectories. The C64 itself knows absolutely nothing about the structure of the DOS syntax or its available commands. SD2IEC is a modern clone of that DOS command set but with the bonus that it backends not on an old SCSI harddisk with a proprietary CBM-derived file system, but on a FAT16 file system on SD Card media. These are, nonetheless, still accessed by the same KERNAL ROM, with the same IEC bus protocol, over the same serial cables that could also be hosting a regular 1541 disk drive.

IDE64 is different though. It plugs into the expansion port like an oversized cartridge. So, how does the standard C64 KERNAL ROM transfer data over the expansion port via standard calls that typically communicate with devices on the IEC serial bus?

There are other storage devices that connect to the expansion port. The CMD RamLink, for example. It obviously has some magic that makes it work, though not necessarily the same magic, and I would someday love to delve into that. More recently there is also the 1541 Ultimate (1541U). I wrote a fairly lengthy and indepth review of the 1541 Ultimate II+ and its many features. (Some of what I said in that post is already out of date, because the firmware in the 1541U continues to get updated.) But while the 1541U is an oversized cartridge, it also has a cable that connects it to the IEC serial port. It's not a huge mystery therefore how the computer is able to see it as a 1541 drive.

How IDE64 works is very clever. And sometimes you need to take a step back and just admire a thing's cleverness. Also, being able to understand how it works is a great example of the compounding nature of knowledge.

I'd long ago heard about the GAME and EXROM lines on the expansion port. I also knew about them from the port expanders, like the CMD EX3 and EX2+1, (now available as the updated and improved X-PANDER 3 from Retro Innovations.) But I never knew what they were for, until I started doing my reading for figuring out how Versa64Cart works. In a nutshell these two lines are inputs from the expansion port to the PLA. Typically a C64's 64K memory map is already overloaded. There is 64K of RAM, plus the 8K KERNAL ROM, the 8K BASIC ROM, a 4K character set ROM and a 4K region of addressing dedicated to I/O devices.

Configurable inputs to the PLA instruct it to change which device or chips are visible to the CPU when one of sixteen 4K blocks of memory are being addressed. I'll reproduce here a segment of the C64's schematic which I originally used in the Versa64Cart post.

The PLA is essentially a 64KB ROM chip, with certain strict timing properties. The 16 inputs on the left side function as the address selector, to produce a custom byte on the 8 output lines on the right. Four of the inputs are from the 4 highest bits of C64's address bus. This is what gives the PLA control over blocks of address space to a precision of 4K. (4 bits = 16, or 0 to 15. 64K / 16 = 4K blocks). GAME and EXROM don't do anything on the standard C64 mainboard except run from two of the PLA's inputs straight to the expansion port. They have pullup resistors so the PLA reads them as high when they are left floating, such as when nothing is plugged into the expansion port.

Typically, a cartridge with a ROM chip in it, connects the GAME and/or EXROM lines to GND to pull them low. This informs the PLA that a ROM cartridge is plugged in. When the CPU attempts to address the memory block where the ROMs ought to reside, the PLA's outputs disable RAM and the BASIC and KERNAL ROMs, and the I/O chips. The external ROM chip is enabled via the ROML and ROMH lines, which run from the PLA's outputs to the expansion port. This is the essential mechanism by which a C64 can already have a full memory map, and yet also have room for a game ROM when a cartridge is plugged in.

This is the stuff I learned when learning about Versa64Cart, and this is mostly what we need to know to understand the basic mechanism by which IDE64 works.

1541 Ultimate II+ vs. IDE64 Hardware Comparision

Before proceeding, I want to talk briefly about the difference between how an IDE64 and a 1541 Ultimate II+ work.

The 1541U is connected both to the expansion port and to the IEC serial port. But, believe it or not, it can be used even when it's not plugged into the expansion port at all.² (With a reduced feature set, obviously.) There is a small USB port on the side of the 1541U that can be used to power it. And then, the computer can communicate with it over just the IEC bus. In this arrangement, it is obvious that the 1541U is its own computer. There is nothing alarming about this, all Commodore disk drives are their own computer. An ordinary 1541 has its own 6502 CPU, a couple of VIAs, 2K of RAM (expandable), and software on its own ROM chip. Together, these know how to talk the IEC bus protocol. The C64 and a 1541 are essentially two independent computers talking to each other over a cable. So it is with a 1541 Ulimate.

The 1541U, however, being far more modern, is unbelievably more powerful than either a 1541 or the C64 itself. And thus, when the 1541U is connected to the C64's expansion port it remains a full and powerful, independent computer. It uses the expansion port to give it more communications opportunities with the C64, and the ability to behave like various expansion port devices (such as a GEORAM, a 1750 REU, a SwiftLink, etc.) When you send a command to the 1541U, either via the IEC bus, or via its own command interface that is mapped into a few I/O registers, the powerful computer inside the 1541U goes off and does its own work. It's executing its own instructions, in its own processor (which is not an 8-bit processor and is in no way derived from or related to a 6502, more about this in a moment), with its own busses and its own RAM before sending out signals allowing it to communicate back to the C64.

The IDE64 is a much simpler beast, but in some ways more authentically retro. The 1541U is built around the rather powerful Cyclone IV FPGA from Intel. On the other hand, according to the components list on C64-wiki, the IDE64 has no FPGA and no microcontroller. Its big component is an M4A5 CPLD.

Here's what IDE64 is made of, more or less:

• M4A5-128/64 CPLD (Complex Programmable Logic Device)
• 128KB CMOS EEPROM (Electrically Erasable, Programmable Read-Only Memory)
• 32KB CMOS Static RAM
• tristate octal line driver (X3)
• tristate bus tranceiver

The only other two chips on the board are an RTC and a USB to Parallel controller to facilitate the PC-Link over USB feature.

The main heart of the IDE64 is a firmware EEPROM, some onboard RAM and a CPLD. CPLDs are mainly used to implement—in a single programmable chip—what would otherwise be made of discrete logic across a bunch of 7400 series logic ICs. Here's what HACKADAY says about the comparison between a CPLD and an FPGA:

FPGAs are better known than CPLDs, but they share many characteristics. This analogy isn't perfect, but we like it: where FPGAs are a reprogrammable processor core, a CPLD is a reprogrammable circuit board or breadboard. FPGAs replace microcontrollers, memory, and other components. CPLDs absorb logic ICs, and work well with a microcontroller. HACKADAY – How-To: Programmable Logic Devices (CPLD) – 2008

So, while an FPGA is meant to implement an entire computer, a CPLD is generally meant to reduce a circuit board full of discrete logic chips down to one cheap and programmable chip. What gets implemented on a CPLD is generally meant to be used with a microcontroller.

But the IDE64 doesn't have a microcontroller, so, how does this thing work?

Putting the pieces together

IDE64 maps itself into I/O 1, that is the region dedicated to expansion I/O devices from $DE00 to $DEFF. When the C64 is configured for I/O to be mapped in (via the processor port, you can read more about how this works in the post The 6510 Processor Port,) and the CPU is addressing between $D000 to $DFFF, the PLA activates a pair of 2-bit decoders. The extra decoders subdivide the 4K I/O region and alternatively activate the VIC, the SID, color memory, CIA 1, CIA 2, I/O 1 or I/O 2, depending on the next most significant 4 bits of addressing.

Everything I'm about to say about I/O 1 could be true of I/O 2 for a device connected to I/O 2, but let's just stick to I/O 1 since that's where IDE64 maps itself.

When I/O 1 is activated, as far as the computer is concerned, this consists of only 2 things. The PLA deactivates every other chip on the bus and activates the I/O 1 line. Typically what would happen is that an I/O chip inside a cartridge would have its enable line connected to the I/O 1 line on the expansion port. All reads and writes that the CPU makes to those addresses are thus routed to that external chip.

This is pretty cool all on its own. This is how you can have a UART highspeed serial controller in a cartridge. Or a second SID chip in a cartridge. Or an ethernet controller chip in a cartridge. Or a midi controller chip in a cartridge. Or a RAM expansion controller in a cartridge. Very clever.

But IDE64 does something much more clever. It mixes both code and I/O together inside that I/O 1 page.

Many of the KERNAL calls, each of the ones listed above at least, are routed through RAM vectors. At reset, IDE64 configures the vector table such that the above vectors point to inside I/O 1, to somewhere from $DE00 to $DEFF. The KERNAL is only ever mapped in when I/O is also mapped in, so if a KERNAL call can safely be made, it's also certain that calls can be made into $DExx.³

Typically code is not executed inside $DExx. But there is no reason why it couldn't be, if code were assembled to run from $DExx and was made available via an external chip when I/O 1 is enabled. The problem is that there isn't a whole lot of room in I/O 1 for implementation, only 256 bytes.⁴ So this is where IDE64 get's really clever. The vectors direct the CPU to run code out of $DExx, but that code tells IDE64 to patch itself in. IDE64 then on-the-fly activates the GAME and EXROM lines, causing some portion of its firmware and RAM to be mapped into the 16K region from $8000 to $BFFF. Then the routine from $DExx can jump into the more extensive IDEDOS routine there.

When the IDEDOS routine is complete, its RTS will return to somewhere in $DExx, which can have IDE64 patch itself out, by deactivating GAME and EXROM, restoring the CPUs access to main memory from $8000 to $9FFF and the BASIC ROM from $A000 to $BFFF. Brilliant.

And thus, very unlike the 1541U, (or even other standard IEC drives) where code is being executed by the drive's own CPU, everything that's in that 128K of IDE64 firmware is actually being executed by the C64's CPU. The C64's 6510 becomes the IDE64's controller. So clever.

Overcoming the Gotchas and Limitations

There are some limitations that popped up as a result of how this works. And these were issues that I needed to work through (with much appreciated help from Leif Bloomquist) in order to make C64 OS compatible with IDE64.

While the IDEDOS is patched in, and the C64's CPU is executing code between $8000 and $BFFF, there is no reason why an interrupt could not occur. It might be possible for IDE64 to disable interrupts, but if it isn't necessary it shouldn't do that, and it doesn't. When an interrupt occurs the CPU is ripped away from whatever it's doing, and jumps through the hardware IRQ vector handled by the KERNAL ROM. It essentially must be handled by the KERNAL ROM, too, it can't ever by handled by RAM underlying the KERNAL, because while GAME and EXROM are active, and CART ROM LO and CART ROM HI are mapped into $8000–$BFFF, the KERNAL ROM is also always mapped in.

Thus the first IRQ handler is the KERNAL ROM's, and it is going to pass control through the KERNAL's IRQ vector at $0314, like it aways does. But here's where things get tricky. You cannot have the IRQ vector pointing anything between $8000 and $BFFF. If your IRQ handler is under there, then when IDEDOS is patched in and an interrupt occurs, it won't find your handler, it will jump to some random place in IDEDOS! And this is exactly what was happening at first in C64 OS.

C64 OS's KERNAL is broken down into 10 modules that are installed from $A000 to $CFFF. (Typically C64 OS has the BASIC ROM mapped out.) Naturally, the service module, which contains the C64 OS IRQ handler, was assembled to somewhere between $A000 and $BFFF. Additionally the service module makes calls to the input module to scan the keyboard and make calls to other input device drivers. And the input module was also somewhere under the BASIC ROM.

The work around here was to rearrange the C64 OS KERNAL modules so that service and input are higher in memory, and fall somewhere between $C000 and $CFFF. I ran into a problem because these modules were unable to fit into the 4K they have to fit into. But this spurred me on. The splitscreen code (read more about that in the post, Raster Interrupts and Splitscreen,) was all hardcoded into the service KERNAL module. This was always a bit dirty from the start, because it takes up at least 700 bytes, and many applications don't actually need to make use of splitscreen.

So, the splitscreen code has now been factored out into a runtime loadable, relocatable graphics library. I'm quite proud of this. That frees up almost a kilobyte of space in the KERNAL, freeing up more main memory for apps that don't need bitmap graphics. And it also allows those KERNAL modules involved in IRQ handling to fit into $Cxxx, and be compatible with IDE64.

Along a similar vein, there is a limitation in where you are allowed to store a filename to be opened by IDE64 or a command string to be sent to its command channel. Whenever you want to open a file by name, on any device, the filename must be in memory somewhere. Then you call the SETNAM KERNAL routine. This configures zero page pointers to the filename. When you call OPEN, the open routine at some point needs to send the filename to the device, and so reads the name from memory by indirect indexing through the preconfigured zero page pointer.

Once again, you have a limitation. Imagine the following. You put the filename in memory between $8000 and $BFFF, then you call SETNAM, and the pointers are configured. Then you call OPEN. This is routed through the OPEN vector to $DExx, which maps in the IDEDOS. It then tries to read the filename through the zero page pointer, but it reads in some portion of its own DOS instead of the filename from the underlying RAM!

So, what happens in C64 OS? As I said, the KERNAL runs from $A000 to $CFFF. The paged memory allocator is configured to allocated memory from $0900 to $9F00. But it hands out pages from the top down. So the very first page it hands out is $9F00–$9FFF. This gets configured as a C64 OS file reference, and boom and the filename is under where IDEDOS is patched into. That's a problem.

At the moment, I don't have a good solution for this. One possible solution is to hint to the memory allocator what the memory will be used for. If it will be used for a file reference the memory allocator could simply ignore free memory higher than $8000 and just look for the first free page lower than that. But, the hinting would make memory allocation more complicated for everything else. That seems to disqualify it as a solution. In the interim, for testing, I've set the memory allocator to only ever hand out memory from $7FFF down. This is also not great, because it wastes a couple of kilobytes of memory. But, it is an easy solution for testing until something better can be thought up.

Other limitations involve direct calls to the KERNAL. If you call the OPEN KERNAL routine directly, rather than going through the KERNAL's jump table, it will bypass the RAM vectors and the IDE64 routines at $DExx will never get a chance to intervene.

This, however, is not a problem for C64 OS. I knew about this issue and avoided doing this sort of thing from the start. Bypassing standard jump tables is not a good idea, even though you'd be tempted to do it to get a bit of extra speed. Especially with repeated calls to CHRIN or CHROUT.

Similarly, if you use the IEC specific KERNAL routines, such as TALK, TALKSA, UNTLK, ACPTR, LISTEN, SECOND, UNLSN or CIOUT, they're going to communicate directly with the IEC bus and have no intervening vectors. You definitely can't use these with IDE64. But there is still a way to use these safely in your routines in C64 OS and maintain compatibility with IDE64.

Normally drive detection is a bit of a pain in the butt. It's not exactly hard, it's just easy to be lazy and not bother with it. Fortunately, C64 OS does drive detection for you. And the only thing it leaves in memory is a small table that specifies the device type codes detected for each device number. Here's how easy it is for a C64 OS application to "detect" an IDE64.

ldy currentdv
lda drivemap,y
cmp #devide64

It's so friggen easy to check if the current device is an IDE64, there is no excuse not to.

Last but not least, for some reason that I can't explain, IDE64 cannot load a file directly to memory lower than $0400. As chance would have it, of course, C64 OS was doing just this during the boot sequence.

In the settings folder is a list of, so called, components. They're not KERNAL modules and they're not drivers or libraries, nor are they apps or utilities. So I called them components. For example, the exception handling code is assembled to somewhere way down low and is listed in the list of components to be installed. The booter just loads in each of the files listed in the components file, in the order they're listed, to wherever they've been assembled to.

To work around this limitation, components are now loaded to the middle of memory first. Their load address is observed and recorded manually, and after being loaded in, the booter copies the component down to where it's supposed to be. It's slightly slower, but it works for all drive types, which is handy because the components get loaded in so early, it's before any drive detection or any of the C64 OS KERNAL modules even get loaded.

Final Thoughts

There was one other small change that had to be made, around the order of switching the IRQ generator from the default CIA 1 timer to the VIC-II raster. Still not sure why that made a difference, but C64 OS is now booting from an IDE64 v4!

How is it? Well, I've seen it booting up on Leif's IDE64 over a video call, and... it's bitchin' fast!

I love it when old hardware makes me go, Whoa! Holy crap, that's fast!

I've got some more posts in the hopper. A discussion of REU programming, and the lightweight REU API in the C64 OS KERNAL. A programming theory post on natural order string comparison in 6502. Multi-page relocatable libraries and porting the I2C6502 library to C64 OS. And more!

1. Actually, the Lt. Kernal harddrive is the true grand-daddy. But, these were very expensive and very rare. In all my days and all my travels I've never even seen a Lt. Kernal harddrive. My understanding though is that it too, like IDE64, connects to the expansion port. [↩]
2. You might think, okay, but if it's not hooked up to the expansion port, how do you get access to its menu system to select and mount D64 images? You can connect its ethernet cable to a network, and you can telnet into its menu system from another computer. You could probably even telnet in from another C64! :) [↩]
3. When the KERNAL is mapped in, it is possible that I/O has been swapped out for the Charset ROM, but when this is the case, it isn't safe to call the regular KERNAL. So it is still the case that when it is safe to call KERNAL routines it is guaranteed that the IDEDOS routines will also be available at $DExx. [↩]
4. With something like a GEORAM, you could easily have code running from $DExx. Since I/O 1 becomes a window into single pages of GEORAM. You could assemble some code to $DExx, store it in the GEORAM and run it directly from the I/O page. But that's unusual. I think typically you would copy multiple pages from the GEORAM's I/O window to somewhere else in memory, and then execute the whole thing from main memory. [↩]

Do you like what you see?

You've just read one of my high-quality, long-form, weblog posts, for free! First, thank you for your interest, it makes producing this content feel worthwhile. I love to hear your input and feedback in the forums below. And I do my best to answer every question.

I'm creating C64 OS and documenting my progress along the way, to give something to you and contribute to the Commodore community. Please consider purchasing one of the items I am currently offering or making a small donation, to help me continue to bring you updates, in-depth technical discussions and programming reference. Your generous support is greatly appreciated.

Greg Naçu — C64OS.com

Want to support my work?

C64 OS version 1.04 is now available

The best way to help C64OS.com grow and continue is to purchase a commercial copy of C64 OS version 1.04.

Join the growing C64 OS user community. Take advantage of the powerful and useful features available now, and get access to run new and future software made for C64 OS.

Learn More Order Now

C64OS.com is hosted on a Digital Ocean droplet.

Digital Ocean is a cloud hosting platform, built for developers. They make it simple to get started. They have intuitive administrative tools and reasonable prices. And they can scale as your needs grow.

If you sign up with this referral link, you'll get a $100 credit, and after you've spent $25, I'll get $25. If you are looking for a cloud platform to host your website or web app, this is win-win.

Sign up for Digital Ocean with this refereral link.
Save money and help C64OS.com at the same time.

No, thank you.

Want to support my hard work? Here's how!