Introduction to File Manager

File Management is considered to be one of the most central parts of an operating system.

In Command Line Interfaces, such as a Unix/Linux or the AmigaDOS Shell, you are in a current working directory. Commands interact with the file system relative to where you are.

In macOS, the Finder is where you find applications, documents, and other files by navigating the file system.

Windows 95, and up to the present day, follows the Macintosh model. The desktop is a folder where you can put other files and folders. And Windows Explorer is the analog of the Macintosh Finder.

Earlier versions of Windows, though, had a Program Manager with icon shortcuts to applications. In addition to this it had a File Manager application for working with files.

For reasons that relate mostly to limited RAM—but also because of the low screen resolution—C64 OS is a bit more like this early Windows model.

Brief Overview of the C64 OS Model

The App Launcher has 5 configurable desktops, but the desktops don't show regular files, they have special desktop aliases to Applications, Utilities and non-C64-OS programs. From the App Launcher, under the Go menu, along with options to move between desktops, there is an option to switch to the File Manager. The App Launcher saves its state and quits, and the File Manager is launched.

From the File Manager you can navigate the devices and their file systems. If you navigate to the Utilities directory and double click a Utility, it opens concurrently with the File Manager. Navigate to the Applications directory and double click an Application to launch it. The File Manager's state is saved automatically.

When you quit an Application, you are returned whence you came. If you came from the File Manager, to the File Manager you will return. If you came from the App Launcher, you return to the App Launcher. When returned to the File Manager, it restores its saved state so your tabs, settings, and places within the tabs are right where you left off.

Along with options to take you to common places in the file system, the File Manager's Go menu has an option to swich you back to the App Launcher. These have keyboard shortcuts so you can easily jump back and forth between App Launcher and File Manager. Thus, they work somewhat like the division of labor in the Program Manager and File Manager of early Windows versions.

App Launcher and File Manager are Homebase Applications. One of them is always configured as your current Homebase. This is why when you quit an Application, you are returned to the Homebase Application whence you came. Both App Launcher and File Manager provide options to Quit to BASIC. The next time you launch into C64 OS, you are returned to the Homebase Application you were last in.

Let's get into some updates about how the C64 OS File Manager works.

File Manager

Now we know a bit about the general model, let's take a look at the user interface of the File Manager.

This is obviously an early build, what with the "Hello World!" filling the list of Favorites. Some of these screenshots are already out of date because I'm working on this code pretty much every night. For instance, the column resizer. Instead of being the diagonal lines it's now the same icon as the splitter's grippy, three vertical lines. Additionally, there is another column in the table now, showing the file's locked status.

Changes like this will continue with each new coding session, but this is nonetheless a close approximation of what version 1 will look like.

I still think the menu bar was an incredible piece of UI invention. This quote from Jack Wellborn in 2018, is spot on so let me repeat it here:

The menu bar has been, and in my opinion remains, the best mechanism for providing familiarity, discoverability, and progressive disclosure in user interfaces on any platform. Jack Wellborn — WormsAndViruses.com, March 2018

I agree. Despite many modern attempts, nothing has yet bested the basic old menu bar. In fact over the last few years, iOS has begun reintroducing standardized menus.

In C64 OS the menu bar is part of the system and so it's part of every application. Creating the menus, their hierarchical organization, and assigning keyboard shortcuts was the easiest part of writing the File Manager. It's just a human editable text format.

Meanwhile, a lot of functionality is available in those menus without taking up any extra screen real estate. Which is important on a screen that's only 320x200. And lest you think, "Yeah, but, the menu bar itself takes up space." With a simple global keyboard combo the menu bar can be toggled on and off. This sends a message to the application indicating a change of system redraw flags. The application has to support this message, of course, to take advantage of it. When the menu bar (or the status bar) is hidden, the top and bottom offsets of the root toolkit view are adjusted. The whole UI stretches dynamically to fill the available space.

In addition to just saving screen real estate, and rendering super fast so you can get at the options in them without delay, they also host discoverable keyboard shortcuts. If you support the menu action, you support its keyboard shortcut too without any additional coding effort.

Many C64 tools and utilities support only a single directory at a time. The more advanced ones usually support two directories. The point of two directories, typically, is to point each one at a different place, select files from one and copy them to the other.

The C64 OS File Manager supports 4 active directories, via 4 tabs. Tabs seem to be in vogue these days, and they also conserve screen real estate. There are menu options, under view, to change tab. Since these have keyboard shortcuts, you can push COMMODORE+1, COMMODORE+2, etc., to toggle between directories quickly.

Directories require memory. Each directory entry requires 1/8th of a 256-byte page. After a page is filled, if another entry is read in (by the directory library) it allocates a new page and links the previous page to the new page. Where the new page is allocated from is dynamic. The pages that make up a chain of directory blocks are not necessarily contiguous. In other words, you don't lose memory to fragmentation. Nor, as the application developer, do you have to worry about that. The directory library and the memory KERNAL module handle all that for you.

Still, 256 directory entries / 8 per page = 32 pages of memory. 32 pages * 4 directories = 128 pages. 128 pages * 256 bytes = 32KB or half of the C64's total memory. But of course, the OS code, the screen memory, the memory reserved for Utilities, the toolkit classes, the application's code, etc., eat up their own memory. There is less than 32KB remaining for directories when the File Manager is running.

What happens if you're in tab 1, navigating the file system, and you click into a long directory and as it's loading entries and allocating new pages, you run out of memory? File Manager keeps track of the order that you've accessed the tabs. It will automatically deallocate the least recently accessed tab in order to free up memory. If it runs out of memory again, this process will repeat, potentially freeing all 3 of the unfocused tabs.

One of the worst offences against productivity is the loss of state. C64 OS provides mechanisms to preserve state wherever possible. Each of the 4 tabs has 4 bodies of information associated with it when it is in focus.

• Directory Metadata
• Place File Reference
• Directory Entry Chain
• Sort Index

The sort index is the most ephemeral. Each time you switch tabs, the sort index is blown away, and reconstituted from the directory entry chain and directory metadata of the tab that comes into focus.

The directory entry chain is the next most ephemeral. It gets loaded in from disk when it's needed, and remains in memory until the memory is needed for something else. As described above, the directory entry chain of a background tab could get expunged from memory to make room for the tab in focus. And it will be loaded back into memory automatically the next time it's needed.

The place file reference is a C64 OS file reference that defines the absolute location of a tab: device #, partition #, filename (optional), full path from root of partition. The tab's place file reference is serialized and written to the File Manager's application bundle. It's clever about when to do this. It keeps track of whether the File Reference is dirty (whether it's changed), and when you switch tabs a dirty File Reference from the tab you're leaving is written out. When a tab is expunged from memory, its File Reference is expunged as well. When you switch to a tab, if its File Reference isn't in memory, it is read back in and unserialized automatically. This is handy for saving memory when switching tabs, but it also preserves each tab's place whenever you leave the File Manager.

The directory metadata is less than 64 bytes. One page is allocated to hold the metadata for all 4 tabs. A tab also tracks whether its metadata is dirty, and writes out changed metadata to disk when the tabs are changed. Similarly, metadata is restored when you first access a tab. However, once tab metadata is loaded into memory, it doesn't get expunged. Doing so would not free up a full page of memory.

Each tab stores and restores the state of its metadata, but what is this metadata? It is a combination of fields that are used by both the File Manager's UI and the Directory Library. Here's the structure of the metadata:

The Directory Header, Directory ID, Blocks Free and Partition Number are parsed out of the directory listing. These can be displayed in optional user interface elements.

The File Count is computed as the directory entries are loaded in, and can be used to display the number of files in the UI and is also used by the TKTable class to indicate how many rows of data it must render, and thus how many rows the TKScroll must allow for scrolling.

The File Pattern Match and File/Partition Type Match are configurable by the UI and are passed through to the device's DOS to search and limit the directory results.

The Sort Field and Sort Options are used by the Directory Library to build the sort index and also used by the File Manager's UI to indicate the sorted field, sort direction and other options.

The Toolkit Advantage

A great deal of the File Manager's functionality is tied up in external resources, as it should be. The File Manager's own code consists of its main binary that is only the business logic that ties those resources together, and a temporary init binary that loads the external resources in, instantiates and wires together the Toolkit classes, and connects them to their delegates and callbacks.

A whole lot of functionality comes in particular from the Toolkit and its classes. For instance, it was mentioned earlier that the menu and status bars can be toggled on and off, and that the UI stretches dynamically to fill the available space. This could be done manually, but it would be a huge pain. It is made absolutely trivial by the way the Toolkit UI objects naturally anchor, resize, hit detect, and hierarchically pass messages, events, etc.

Let's talk about some advantages the C64 OS Toolkit provides to the UI.

Split View (tksplit)

The root view is the vertical split view. It creates the bar that separates the directory listings on the right from the device and favorites navigation on the left. You can drag the splitter left and right to decide how much screen real estate you want to allocate to each side.

Speaking of persistence of state, the File Manager stores in its general config the position of the splitter. You can drag the splitter all the way to the left to hide the side bar completely, and when you leave and return to the File Manager later, this preference is remembered.

Why would you want to allocate more space to the directories? So that they can show you more columns of data on a single row. We'll return to this. The fact that you can move that splitter around is extremely rare in ordinary C64 programs.

Places View (tkplaces)

TKPlaces is a custom class. It's not memory resident when an application is running that doesn't need it. The File Manager's init binary loads and relocates it to any available spot in memory.

Also, I'm coding everything native, on the C64, using Turbo Macro Pro. TMP is powerful and offers lots of features, but it's not unlimited. The assembler has to work within the memory limitations of the C64. Carving functionality off the application's main binary and into neatly contained Toolkit classes, whenever this is possible, is a major boon to development.

TKPlaces uses inheritance and composition. Much ink has been spilt on this topic. It subclasses TKView, and does some primitive drawing and event handling. But it embeds a TKScroll and TKList for the favorites in the bottom section.

What's nice about this is that the logic for drawing the current devices, and folding up that section by clicking its section header, and showing the favorites list, etc., is all in a separate source code file. It feels like really good coding practice, but on the C64 with Turbo Macro Pro it's more than just a good practice, it's essential. If I tried to pack all of the functionality of the TKPlaces class directly into the File Manager's main binary, TMP would run out of memory and wouldn't be able to assemble it all.

The way that Toolkit classes plug together and communicate with each other is pretty great.

Tab View (tktabs)

The TKTabs view is a built-in class. It gives you up to 10 tabs. To use it you just append from 2 to 10 child views. They automatically have their metrics handled for you. Each child is set to fill the tab view's area, but are offset from the top edge by one row, to make room for the tabs themselves. This would cause each child view to overlap perfectly with all the others, but the TKTabs marks all but one as hidden. View hiding is a feature of TKView, from which all the others descend.

The TKTabs view has a delegate property, which gets pointed to a structure in the File Manager's main binary. It has routine pointers for Tab Title, will blur, will focus, did blur and did focus. Whenever the tabs need to redraw they call the delegate to fetch the titles for the tabs, so the File Manager has complete dynamic control over what to show on the tabs. And when the user clicks on a tab, the sequence is followed: current tab calls will blur. Will focus is called with an argument for the tab index that will focus. Did blur is called on the current tab before switching tabs. Tab is switched, and did focus is called with the new current tab.

Either the of the will routines can be denied by returning with the carry set, and abort the sequence. But once will blur and will focus are allowed to occur, the sequence can no longer be aborted and the did routines are there only for notification purposes. The implementation of this by the TKTabs class is very short. And if you don't want to support these in the delegate structure, you can just populate it with generic routine pointers like this:

tabsdel
     .word tabtitle
     .word clc_rts ;will blur
     .word clc_rts ;will focus
     .word raw_rts ;did blur
     .word raw_rts ;did focus

But what's really great about these is that they give complete control over to the application for what's going on and what's allowed to go on. If you are in an inconsistent state in the current tab that needs to be resolved before leaving this tab, you can implement will blur to check for that state and deny the tab change.

In the File Manager, the user can change tabs whenever they want, but did blur is used to save the metadata and place of that tab if they're dirty. And did focus is used to change the metadata pointer, and then call resort on the directory, or, if the tab's content isn't loaded in, to call reload.

Table Columns View (tktcols)

The table view, while very common in UI's, I had to create new for the File Manager. I was a bit worried about how to implement it, would it take up too much memory, how would it even work? And it turned out to be remarkably easy, given what was already available. It's made of two classes, each a subclass of something that already existed. TKTCols is for the table column headers, and TKTable for the table body.

TKTCols is a subclass of TKScroll. TKScroll subclasses TKView and allows you to append a single scrolling child. In addition to that child, you can turn on the horizontal and/or vertical scrollbars. Each scrollbar is an instance of TKSbar. TKScroll creates and destroys the TKSbars for you, and appends them as children of itself, siblings of the one child view you're allowed to append to the TKScroll. When a horizontal scrollbar is enabled, the main child view has its offset bottom set to 1, to make room for the scrollbar. And when the vertical scrollbar is enabled the main child view has its right offset set to 1, to make room for the vertical scrollbar.

The TKScroll then liaises between the three children. If you move the scrollbar the scrollbar tells its parent, and the parent reads its new value and updates the main child's scroll offset for that axis. If the child view has its scroll offsets changed (due to being interacted with, or because its total content changes size) the TKScroll sends the child's new metrics to the TKSbars and they rerender with new proportions and new offsets. Beautiful. Okay, but that's how TKScroll works. What about TKTCols?

The column headers, somewhat like tabs, have to appear in a strip along the top of a table of rows and columns. The table rows have to scroll up and down, but we want the column headers to stay in the top row. But, we also want the table's columns to scroll left and right, so we need the column headers to scroll left and right too. The column headers can't be inside the scroll region, and on desktop OSes they're generally not. As in the example below, from macOS Big Sur, when you scroll vertically, the content in the green box moves up and down and the top content in the yellow box stays put. When you scroll horizontally, though, the content in the green box moves left and right and the content in the yellow box scrolls left and right synchronously.

It doesn't seem like a big deal, but it does require some thought. TKTCols subclasses TKScroll and then, similarly to the TKTabs class, it sets the top offset of both the main child view (the equivalent of what is in the green box above) and the vertical scroll bar down by one. Miraculously, the proportionality of the vertical scrollbar is completely okay with this. It figures out how big and where the scroll nub should be by translating the child view's metrics against its own metrics. In other words, the scrolling child view could be 18 rows high with 200 rows of data, and the vertical scrollbar could be only 15 rows high (say to make room for some extra controls above or below it), and yet the nub inside the scrollbar will still correctly represent and control the scrolling of that child view. That is damn cool.

With the child view and the vertical scrollbar offset down from the top by one row, there is a blank space into which the TKTCols can primitively draw the column headers. The horizontal scrollbar, though, could be fiddled with which will scroll the child view but must also affect the column headers. The role of the TKScroll class is to liaise, and since the TKTCols is a subclass of TKScroll, it gets the messages from the scrollbar about a change in scroll offset. It sets this offset in a special column-headers-left-offset property, marks itself dirty, and then calls its superclass implementation which forwards the message to the child view. Remarkably little code is necessary to do this.

Because the TKTCols is marked dirty, it will be told to redraw. Whenever it redraws, its primitive drawing code that draws out the column headers takes the column-headers-left-offset property into consideration. And boom, the column headers stay synchronized with the table columns below.

Table View (tktable)

TKTable is a subclass of TKList. TKList, I created not long ago for use in the Utilities Utility, which you can read about in detail in the post, Anatomy of a Utility. TKList supports a single scrollable column of content. It gets its content, including row count, column width, string content per row index, as well as row selection status from a series of callbacks.

TKTable inherits most of the behavior of TKList, but extends it to multiple columns. To do this it adds a property that points to a set of column definition structures. When getting the row count (the vertical content size) it continues to use the callback via TKList's implementation. But when getting the content width, rather than using the callback it just sums up the current widths of all the column definitions. Here's what a column definition looks like:

This 8 byte structure can be repeated multiple times, contiguously in memory, to define more than one column. The set of all column structures is terminated with two $00 bytes. Any single byte value could be interpreted as the low byte of the tc_name pointer of the next column. But 2 $00 bytes makes the string pointer $0000, which is not valid, and hence a good terminator.

Now here's what's kind of neat. The TKTCols object and the corresponding TKTable object are both pointed to the same set of column definitions. The column title is not used by the TKTable, but is used by the TKTCols. The alignment is not used by the TKTCols, but is used by TKTable. The TKTCols uses the resizable flag to know whether to draw the column resize grippy. It could compute this by checking to see if min size and max size are the same, but that takes longer to compute, so the resize flag is just to speed up rendering, and make the code simpler.

The TKTCols use the resize flag and the min and max sizes to allow the user to drag the column wider or narrower, within the min and max limits. It adjusts the current size, then marks itself dirty. When the TKTCols and TKTable redraw themselves, they both use the current size to know where to cut off the content.

TKTable uses the same content at index (ctntaidx) callback property that TKList defines, but in its subclassed implementation, instead of calling it once for every row, it calls it once for every row/column combination, passing the row and column index in the Y and X registers respectively.

Remember, the custom classes created for File Manager are usable by any application. The TKList, TKTCols/TKTable, even TKPlaces can be used in other Applications and Utilities. These will all be used in the File Open/Save Utilities.

Common File Management Tasks

There are several common functions of a File Manager:

• Navigate the file systems (partitions and subdirectories) of multiple storage devices
• Open an Application or Utility directly
• Open a file which indirectly opens a corresponding Application or Utility
• Create a new subdirectory
• Rename a file or subdirectory
• Scratch one or more files
• Recursively scratch one or more subdirectories and files contained therein
• Copy one or more files, in place
• Recursively copy one or more subdirectories and their files to a different path, partition or device
• Recursively move one or more subdirectories… aka, a recursive copy followed by a recursive scratch

Directory Library

The directory library has everything necessary to load any directory (including a partition directory) from a pointer to a standard C64 OS File Reference. Navigating the file system, then, is a matter of manipulating the data in the file reference structure, and telling the directory library to reload.

There is this thread on the uIEC Users Discussion Group (on Google Groups) about supporting the Unix concept of "PWD" or "Present Working Directory" in the SD2IEC firmware. The thread mostly took place in 2016, and I chimed in a year later in 2017, and by that point no one was paying attention anymore. Here's a clip of my comment that is finally relevant here.

I'm interested in how to implement PWD. And Glenn's promal code is the closest I've come to a fully thought out solution. But, it occurs to me that manually figuring out the PWD on the fly is not as important inside an OS as I originally suspected. [As] the user uses the OS to navigate the file system, the OS merely needs to keep track of where the user has gone as he goes there. Gregory Nacu — 2017 — uIEC Users Discussion Group

This idea pre-dated the development of C64 OS File References, which I first wrote about in early 2018. Because of the C64's limited memory and processing power, it made (continues to make) a lot of sense that the DOS and File System code are farmed out to the storage devices. But it has some disadvantages too and it's a bad idea to rely too much on a device's internal features.

For example, even if SD2IEC implemented a feature that could easily provide you with the present working directory, it's unlikely that IDE64 would follow suit, and the CMD HD DOS is not going to be changing any time soon. A feature that is available on only one device is a feature with limited appeal. We need solutions that work across every device or most devices.

CMD and SD2IEC devices have a concept of a current partition and a current directory within each partition. This is very useful from the READY prompt. You issue a command to change partitions, and then to change directory, and now every subsequent command you issue applies to that partition and directory. You can scratch a file by name, for instance, and the name is searched for within the current partition and path. But the moment you have a more complex system with different parts that are both running at the same time (even without true multi-tasking), you can no longer rely only on those internal device representations.

For instance, your Application wants to read and write files from within its own application bundle. But you close a Utility and it wants to write its state to a config file in the system's global settings directory. If the Utility changed the current directory, the Application would immediately become lost. GEOS has this problem, Wheels took a very small step to try to alleviate this problem, GoDot has this problem, and, well, frankly I'm not that familiar with many other large multi-file systems on the C64. All I know is that relying on the device and some DOS feature to try to figure out post hoc where you are in the file system is a nonstarter.

The point of a C64 OS File Reference is that it provides an absolute location on the device, in a standardized memory structure. And it can also be serialized and unserialized for storage and retrieval from disk. The serialized format is also human readable and human writable. The status bar, for instance, serializes the current open file file reference to display the path to the user. Serialization also opens the door for allowing the user to type a file reference into a text field, which could be unserialized into a memory structure.

The serialized structure looks like this:

dev#:part#:filename:path

Partition # is drive # on devices without partitions, such as 1541/1571/1581, where the partition # would be 0. If the partition # is 0 on a CMD or SD2IEC device, the directory library fetches the partition directory instead. Partition #0 can thus be thought of as a device root.

The filename and path are also allowed to be empty. But if the path is not empty it must be a full directory path in the CMD/IDE64/SD2IEC format. It starts with two slashes (for root of partition), and each subdirectory part is 16 characters or less, plus a trailing slash. Directories on all 3 device types are structured like this, this standard on the C64 was developed by CMD:

//sub directory 1/sub directory 2/sub directory 3/

The following then, are all valid serialized file references:

8:0::
9:1:://documents/
10:15:myfile.txt://
12:0:somefile:

etc.

Path Library

Manipulating a file reference isn't precisely hard, but even trivial things seem to take up a lot of memory when you only have 64K to start with.

Appending a directory name to the end of the path, for example.

• You need a pointer to the source string.
• This involves copying a pointer to some temporary zero page addresses.
• You have to know how long the current path is of the file reference.
• This involves setting up a pointer, and calling strlen in the C64 OS KERNAL.
• You need to copy the bytes from the first string to the path pointer after adding the strlen to the pointer.
• You have to do that because the 6502 can only indirectly index using the Y register. The source string's index will run from 0 to 15, but the destination will run from the length of the path string to the length of the path string plus 0 to 15. It's much easier and faster to add the string length to the destination pointer and then use the same Y index on both the source and destination.
• You have to manually add the trailing slash.
• And Lastly, manually add the new null terminator.

Again, it's not exactly hard, but you really don't want to have this amount of code strewn here, there and everywhere, every time you want to append a path.

The path library makes the common tasks of manipulating file references easy. To go up a directory, you have to remove the final path part. But doing so manually, like appending a path part, is annoyingly verbose. The path library makes it one call. The following routines are provided by the path library:

Of these, gopath is the most interesting, in my opinion. The idea here is to allow a set of single character codes to represent standard places either system- or user-defined. For example: The applications and utilities directories are system-defined, but relative to the installed location of the system directory. Codes such as "a", "u", "s" passed into this routine would configure the file reference for those places.

Additionally, codes for standard user directories such as:

Each of these could fetch and unserialize a file reference from the system settings directory. This would allow the user to custom define these places.

The path library is in progress and is still evolving. I'm considering using this library to also automatically manage favorites and recents. Favorites being a customizable set of places you want quick access to. Recents being the files recently opened or saved, on a per application basis. The File Open and File Save Utilities will embed the path library to help them navigate the file system, so it seems easy to implement and an obvious role for the path library to manage favorites and recents too.

Libraries, Libraries, and More Libraries

I'm writing this post while working on this code at the same time. And I've realized that the set of libraries is growing. Therefore, I've taken a slight detour to standardize the libraries and add proper support for loading and unloading libraries. There is a loaded library table to find their relocated base address. They use reference counting to know how many things are depending on them, so a library loaded once can be shared by the system, the application and a utility.

This is kind of a big side discussion, so I'm going to make it the topic of the next post.

Utilities for File Management Tasks

Simple file management tasks that apply only to one or more file in the current directory will be built directly in the File Manager.

For example, select a single file, select scratch from the menus, and that file will be scratched. Or, pick a single file, select copy from the menus, and that file will be copied, in place, and assigned a new name automatically based on the original's name. Select create directory from the menus, and a new subdirectory will be created with a default name.

Each of these consists of a single command that can be sent to the device's DOS to perform the work. Commands like these, respectively:

s:filename
c:filename 2=filename
md:new directory

But what happens when you want to do something more complicated? Like, what happens when you want to scratch an entire directory tree? There are several problems that need to be dealt with.

1. It's dangerous. The user should have an opportunity to cancel or confirm.
2. It's recursive. It requires a fair amount of code to perform the job.
3. It's long running. The user should see an indication of progress.
4. It's useful, so this feature should be reusable by other applications.

Scratching one file or even multiple selected files is a bit dangerous. You don't want to do it by mistake. To mitigate the risk of doing this accidentally we can: separate the scratch menu option from the others, and assign it a keyboard shortcut with a multi-modifier key combination. If you still manage scratch a file accidentally, you can use an unscratch program to recover it.

Scratching a directory, on the other hand, is much more dangerous. It's going to kick off a process of scratching an unspecified number of files and subdirectories in total. Unscratching might still be possible, but it would be a serious amount of work. Some sort of dialog box that pops up, presents to the user a summary of the dangerous job about to be done, and allows them to cancel or proceed, is almost essential. C64 OS doesn't have a window manager though, and producing and managing dialog boxes is not built in.

Performing the recursive job is nowhere near as trivial as issuing one or more single scratch commands. Writing this code into the File Manager's main binary leads to several problems on its own:

• The code would take up memory space that could be used for holding directory data.
• The code would have to be loaded into memory at the time the File Manager is launched, increasing load time.
• When programming natively, space in a source code file is limited. The main source code file would contain even more code and labels.

After all of this, the user may want to recursively scratch a directory only once in a while. Given that there is no easy way to recursively scratch from the READY. prompt, this is clearly not something a C64 user does very often. It seems almost criminal to load it all into memory every time the user pops into the File Manager to do something.

And lastly, the job itself is long running. File system access is not known to be super fast on the C64. We don't want to just start the job and have the UI essentially lock up and give no feedback until it's complete. Rather, it would be great to show the user that something is happening, and better still to show them some actual progress details or current status. Doing this extra stuff would be even more onerous on the primary binary.

The Obvious Solution: A Utility

The answer is obvious: a Utility. One Utility each for these recursive jobs, one for recursive scratch, one for recursive copy, and one for recursive move. The jobs of move and copy are more work than scratch, because the move or copy could cross devices. A file can be copied from one place to another within one device by sending it a single DOS command. But to copy a file between devices means loading it into the computer from one device and sending it back out to another. There is an opportunity here to use an REU to facilitate and accelerate the copying, but that's more code and more complication.

Offloading this functionality into a Utility addresses every one of the problems.

A Utility:

• Provides a dialog-like window for a message, cancel and proceed buttons.
• Provides a user interface surface upon which to show progress.
• Has separate source code files from the Application's main binary.
• Only gets loaded in when the user wants to perform that job.
• Gets loaded into Utility memory space, which isn't shared with main memory (because of KERNAL ROM contention), so it doesn't deprive the application of memory used for directory data.

And… a Utility can easily be invoked by any application, not just the File Manager. It's a wonderful solution.

Passing Job Metadata to a Utility

A Utility can be opened manually, however, not only by being invoked from another application. Somehow these File Management Utilities have to know that they were opened without any information about a job to perform and present a reasonable message to that effect. Something like:

And then present no other buttons, besides the standard close button. If the user opens it manually that's all they'll see, but at least they know what it's for.

Whenever a Utility is opened, it may check the Open Utility Message Command (opnutilmcmd.) When an application invokes a Utility for a specific purpose it uses this memory location to pass a single command byte to the Utility upon opening. Some commands are implicitly associated with 1 or 2 additional message data bytes, (opnutilmdlo and opnutilmdhi.) For some messages, 2 bytes are sufficient for additional information. If a command needs more than 2 bytes it can define them as a pointer to a structure. The command mc_mptr is a generic message that means, "the message data bytes contain a pointer."

Therefore, each of this Utilities needs to confirm first that the message command is mc_mptr. And after that, each utility will analyze the structure pointed at to confirm that it contains a valid job. Here are the job structures I think these are going to use:

Move

Copy

Scratch

For instance, the Move Utility can check the first 4 bytes of the structure to which it is passed a pointer. If the are not "move", then it can present its standard message as though no job had been passed at all. This is what will happen if that Utility is opened manually.

Once it confirms that the structure has a file operation ID of "move" it can safely proceed to interpret the rest of the bytes to know what to do. All File References in C64 OS are page-aligned. So the source and destination file references only need the page number to be passed in. Additionally, the blocks for directory entries that are chained together by the directory library are also page-aligned. Each directory entry contains a file status byte with bits for: Locked, Hidden¹, Open, and Selected.

The chain of blocks containing the directory entries contain only directory entries. Filenames, sizes, file types, status flags, timestamp information, but not where these files actually come from. The Utility can read through the directory chain searching for entries whose status indicates they are selected. It can then perform the operation on that file by reference to the source and destination file references.

A scratch doesn't need a destination file reference, but the byte offset is left vacant so the offset of the other fields is the same for all of the job structures.

Validated Recursive File Operation

A utility that blindly copies everything, or blindly moves or scratches everything, is useful but is bound to bump up against limitations if it gets used by other applications besides the File Manager. To make these Utilities useful in a wider context, the job structures have a validation callback.

On every directory entry, be it a file or subdirectory, the validation callback will be called with a pointer to the directory entry being processed. The application that supplies the validation callback routine already has pointers to the source and destination file references. The validation routine can do whatever it wants with this combination of information to determine whether the entry should be processed or skipped.

It could, say, skip SEQ files found inside application bundles, if the filename ends in ".i". Just to pick a zany example of the flexibility it makes possible. If the validation returns with the carry set, the Utility will skip that entry. If it's copying, it will not copy that file, or if it's a directory it will not create that directory at the destination, nor recurse into that directory at the source. If it's scratching, it will not scratch that file, or will not remove that directory, or recurse into that directory, and so on.

Many routines in C64 OS use the carry to indicate success or failure, or whether to propagate or stop propagation of an event, or whether a message was handled or not, and so on. There are system constants that point reliably to CLC followed by RTS, and SEC followed by RTS. These are defined, unimaginatively, as CLC_RTS and SEC_RTS. You must provide a validation callback, but if you just want to validate everything, then assign CLC_RTS as the callback. It will be called but it will always return immediately with the carry clear.

Let's take recursive scratching just for an example of how this would work.

It gets a source file reference and a pointer to the start of a directory chain. On its first pass, it's looking for selected entries. On subsequent passes (nested directories) it will apply to all items in the directory. First it initializes the device from the source file reference. This sets the device's internal current directory and partition to the source location. It begins looping through the directory entries looking for those that are selected, and automatically skipping those that are not selected.

It calls the validation callback with a pointer to a directory entry. If the carry comes back set, it skips that entry, and loops to look for the next. If the carry comes back clear, it checks the type of file. If it's not a directory, it sends a DOS scratch command on that filename. The scratch occurs relative the current partition and path that the device has been initialized to. Note that this is an advantage of a non-multi-tasking OS. This whole process is atomic, nothing will change the current directory on the device out from under us.

If the file is locked, and the device (correctly) refuses to scratch the file, that's not an error and is ignored.

If the file type is directory, here's how it recurses: It uses the path library to pathadd the name of the current directory entry to the file reference. Recall that the file reference lives in the Application's memory space, to which we merely have a pointer. It uses the directory library to load a directory from the file reference. This causes the device to be initialized again with the same file reference but a different path, which causes it to change its current directory. Then it recurses by calling the same directory processing loop again. To recurse properly, we'd backup the directory pointer first.

The only difference on this nested pass is that it no longer cares whether the directory entries are selected or not. It's easy to handle this, with a simple variable that tracks nesting depth. It starts at zero. Just before the recusion call the nest depth gets incremented, and just after the recursive call returns the nest depth gets decremented. During the loop if nest depth is not zero it skips the selected status check.

Finally, when we reach the end of looping through a directory's entries, if the nest depth is not zero, we use the directory library to free the directory. And we use the path library to call pathup on the file reference, and then reinitialize the device so its current directory returns to where it was.

After returning from a recursive call it issues a DOS command to remove the directory just processed. If there were any locked files below this directory, then the directory will still contain items and the remove directory command will fail. Just like failing to scratch a locked file, this is not an error, and will be ignored.

When we reach the end of a directory, and the nest depth is zero, we do not want to free that directory because it is owned by the application that assembled the job. Also note that, along the way, the file reference owned by the application has its path changed. But by the time the recursion completes, the file reference is restored to its original state.

Final Thoughts

So that's what I'm working on for the File Manager. But, as I said, I got slightly detoured to implement shared libraries, which will be the topic of the next post.

You might have noticed why shared libraries are suddenly important. The File Manager application loads in the directory library, and the path library, and some other new libraries. But then, to perform some task such as recursively copying a directory, it invokes a Copy Utility. But that Utility needs to make use of the directory library and the path library too! It could load a copy of them, but that's a huge waste of memory on a machine where memory is at a premium.

We'll talk about shared libraries next time.

Hope you enjoyed this. Things are really heating up now. I'm starting to develop things that are not typically seen on the C64. It's exciting.

1. Hidden is a feature of SD2IEC. By default a directory doesn't include hidden files. An extra "=H" can be used include hidden files. The hidden files in the directory listing are marked with an H, which is supported by the C64 OS directory library.

Do you like what you see?

You've just read one of my high-quality, long-form, weblog posts, for free! First, thank you for your interest, it makes producing this content feel worthwhile. I love to hear your input and feedback in the forums below. And I do my best to answer every question.

I'm creating C64 OS and documenting my progress along the way, to give something to you and contribute to the Commodore community. Please consider purchasing one of the items I am currently offering or making a small donation, to help me continue to bring you updates, in-depth technical discussions and programming reference. Your generous support is greatly appreciated.

Greg Naçu — C64OS.com

Want to support my work?

C64 OS version 1.04 is now available

The best way to help C64OS.com grow and continue is to purchase a commercial copy of C64 OS version 1.04.

Join the growing C64 OS user community. Take advantage of the powerful and useful features available now, and get access to run new and future software made for C64 OS.

Learn More Order Now

C64OS.com is hosted on a Digital Ocean droplet.

Digital Ocean is a cloud hosting platform, built for developers. They make it simple to get started. They have intuitive administrative tools and reasonable prices. And they can scale as your needs grow.

If you sign up with this referral link, you'll get a $100 credit, and after you've spent $25, I'll get $25. If you are looking for a cloud platform to host your website or web app, this is win-win.

Sign up for Digital Ocean with this refereral link.
Save money and help C64OS.com at the same time.

No, thank you.

Want to support my hard work? Here's how!