Status on the generic storyline editor

My design needs to deal with the following:

Persistence
Syncrhonisation
Platform dependence
Programming Language dependence
Network dependence
Command response delay
User Interface
Backwards and forwards compatibility

Each area poses a unique challenge. Let's go over them one by one:

Persistence is necessary because an editor by definition needs to produce something persistent. But there are several kinds of persistence, and some are more valuable than others. One type is file serailization; this is a snapshot of the current results that is only readable by the current program and perhaps a hex editor. Another is XML-based serialisation; this allows for synchronization, limited interoperability with other programs, and limited readability by humans by means of standard text editors; but it's still limited to snapshots. The third, most advanced, and best option is persisting every change made in the editor along with a vector timestamp, an internet timestamp (if available), a local timestamp, and a user ID, and storing it as an atomic patch.

Synchronisation is necessary if more than one person uses the editor on the same project, or even if that one person works on the project from several locations. Backup is often considered an intrinsic part of syncrhonisation, because if you have access to synchronisation, you have access to backup.
There's a drastic difference between single-user and multi-user synchronisation; it's generally acceptable to use manual synchronisation between workstations, as well as manual backup, when there's only one user; further, it's also generally accepted that if that one user were to make different changes to his project at two different places without synchronising, all the changes made in one of the places would be lost upon synchronization: this limitation is accepted because a single user can generally "work around" this situation such that it never happens.
But for multi user synchronisation, it's generally necessary to allow simultaneous work in several places at once without having them manually reenter all the changes made in all but one place, in order to synchronize them.
Ideally, multiuser synchronization is either synchronous, or asynchronous but automatically run at set intervals and/or on demand, and when it occours, it usually does all, or almost all of the work without user intervention for solving conflicts.
Typically, synchronization also provides sync reports that go into detail on what was changed with each sync, so as to keep all users up to date. Synchronous synchronization is easy to implement, because everything is sync'ed all the time, but usually lags things down because all user clients need to be constantly chattering about whats happening all over the place, and demands solid network stability to the point of being always-on. Asynchronous sync is much harder to implement, but poses no strict network demands; instead, users must provide ID's, abd must help solve conflicts, but in return they're likely to get a snappy and free user experience.

Platform dependence is really OS dependence these days; all work done is done on PC's, so the only question is if the editor is able to run on all platforms natively, or if it has to happen on virtual machines. Platform independence is generally considered the more future-proof alternative, but it can pose some strict interface limitations because platform independent interface libraries generally look odd on most platforms.

Programming language dependence is closely associated to platform dependence; choosing a language will generally lock you in to using just that one language or having you incur rather tremendous costs, but there are different degrees of lockin. Python, for example, is often the second or third language that gets bindings to access libraries of another language, meaning you generally have several languages to fall back on. Some languages are also able to run on several different virtual machines, or interpreters, which can be a boon for future-proofing. Finally, some languages are exceedingly interoperable, allowing you to abandon them when a better one arises by means of automated translation.

Network dependence is just what it says; will the program need net access to function or not?

Command response delay is really a measure of how snappy the program is. If ever a user has to wait for the program, he becomes a _lot_ less productive; a short delay generally translates to a break in workflow, which makes a 10 second pause translate into several minutes of lost work. Some technologies _cost_ on this front; so it's one of those things that isn't an issue when you start out, but if you make the wrong tradeoff somewhere along the road, suddenly you have a useless program, or one with major drawback.

User Interface is just what it says; there are good user interfaces, there are clunky ones, there are bad ones, and there are ones that only a programmer can use, usually by command line. What type of user interface you need depend upon the target usergroup, and your ambitions with your application. The target audience of this program is fairly broad, since it simply means to get at creative individuals with no other qualifications, and they work better if they have a better interface. The ambitions are also high, of course, so that doesn't improve things. This makes the interface bit quite important.

Backwards and forwards compatibility is a matter of allowing the users to get at the program early while not hanging the sword of damocles over their head: If users are scared that future versions of the program won't be backwards compatible, they won't be inclined to use it.

So, now that we have some design parameters up, let me follow through with what I've been thinking, and how I've planned to tackle these issues so far...:

I've allowed the idea for a generic storyline editor to simmer for a while now, since about december 08 when I last did serious work on it, and have now come back to it.

And when I write idea, I mean plan. Traditionally, it's considered counterproductive to plan too far ahead concerning software development; rather, rapid prototyping and modular approaches tend to make for a far smoother creative process with far less grief over wasted effort.

It could be considered a mistake, then, that I've allowed my idea to linger for as long as I have. Thing is, allowing time to pass tends to reveal obvious design errors. One of which I feel I was close to making.

There is a reason, for example, that everybody hates the object-relational mismatch: The best way to store data is in databases, the best way to program is with objects. It's not just that these two methodologies are somewhat incompatible; it's that in spite of this incompatibility, using the two in conjunction is better than the alternatives for a wide range of software products.

Since a generic editor is by definition going to be a program with general utility, and since the projects meant to be edited by it are huge and may live to see several versions of it, such an editor closely mimics the utility pattern of enterprise software, and here's the point: It's precisely this type of software that usually needs both relational data storage and object based algorithms. Not because there aren't alternatives, but rather because the alternatives have been found wanting by the people making the decisions.

Fortunately, since the information needs to be available as objects in the application, I can start there, and simply be mindful that I only use plain serialization to disk if I need to persist anything at first. This means that the persistant data is, initially, bound to the programming language of the editor, but that's a small price to pay for utility before I have figured out how to write a database management module.

So, that means: persistence is going to be simple serialisation of objects for now, database stuff later, hopefully atomic patch-based.

Synchronisation will be solved by means of the database, again, later.

This means that, later, networking will be necessary for sync, but for nothing else.

This also means that there will be low command delay, since all the information will be available in local objects, always.

I've not made any final decisions on most of the other things, so that's the status for now.