Persistence:
For version 0.1, will be using user-activated object serialisation.
For version 1.0, will be using atomic timestamped patches that are pervasively stored, backed up, and synchronized when possible.
Platform dependence
All versions will likely be platform independent, both with regards to OS and processor. Python and clutter will be utilized for the bulk of the work.
Programming Language dependence
Python will be utilized for main development, allowing for mid-level portability.
For version 0.1, this is not a concern.
For version 1.0, XML export must be implemented and fully supported.
Network dependence
This will not be necessary for any versions. Version 0.1 will not support it.
For version 1.0, it will be necessary for pervasive synchrony.
Command response delay
No delay is acceptable in any versions.
User Interface
0.1: Basic experience with a basic font. Functional auto-placement and folding algorithm.
1.0: Pixel-precise user-customizable fonts, dynamic and fluid interface experience, with panning, zooming and scaling all incorporated, and dynamic folding and placement of everything. Placement structures and colour schemes incorporated to allow writers to use their visual memory.
Backwards and forwards compatibility
0.1: No focus is given here for this release.
1.0: Fully forwards and backwards compatible, as well as allowing for different branches when necessary. A simple tool for merging, splitting, and maintaining synchrony across versions and branches.
søndag den 29. marts 2009
Programming language and graphics API for the generic storyline editor
I've been examining a couple of different programming languages and API's. The first is java and swing; java is the most widely used programming language, and almost always the most efficient one in terms of the number of necessary CPU cycles while maintaining stability.
It is, however, prone to several problems: Java garbage collection leads to annoying "jitter" because it isn't designed for maintaining a smooth user interface. Secondly, java swing looks weird on both windows and nix systems, being visually different from native window rendering.
Finally, java is...well, annoyingly heavy to write.
C# and windows presentation foundation is option number 2...C# is a smaller and less utilized language than Java, but it's only properly supported on microsoft platforms. Windows presentation foundation, while teoretically just what I'm looking for, is currently under-utilized as well. I don't quite dare jump on C# for these reasons...well that, and I don't like being limited to microsofts platform unless I have to be. C# is also a fairly heavy language to write.
Third option: Python and Clutter or Cairo. Clutter is a visual open source API intel is developing that can do all sorts of neat tricks. Python is google's favoured programming language, and they're currently actively developing solutions to it's biggest drawbacks. Cairo is a visual open source api that isn't being shouldered by any specific company, but it's big enough that the mozilla foundations gecko rendering engine, used in firefox, relies on it.
There are bindings from python to both api's, meaning that they should function with it; python is, itself, a language with a fairly large user base (bigger than C# and small than java), and it's dead simple to write and use. Very fast. And Clutter and Cairo look excellent; not necessarily like something that runs natively on any one system, but then, it's so fancy that it looks more next gen than simply different.
I'm hoping to use python and clutter, and if that doesn't work out, python and cairo.
It is, however, prone to several problems: Java garbage collection leads to annoying "jitter" because it isn't designed for maintaining a smooth user interface. Secondly, java swing looks weird on both windows and nix systems, being visually different from native window rendering.
Finally, java is...well, annoyingly heavy to write.
C# and windows presentation foundation is option number 2...C# is a smaller and less utilized language than Java, but it's only properly supported on microsoft platforms. Windows presentation foundation, while teoretically just what I'm looking for, is currently under-utilized as well. I don't quite dare jump on C# for these reasons...well that, and I don't like being limited to microsofts platform unless I have to be. C# is also a fairly heavy language to write.
Third option: Python and Clutter or Cairo. Clutter is a visual open source API intel is developing that can do all sorts of neat tricks. Python is google's favoured programming language, and they're currently actively developing solutions to it's biggest drawbacks. Cairo is a visual open source api that isn't being shouldered by any specific company, but it's big enough that the mozilla foundations gecko rendering engine, used in firefox, relies on it.
There are bindings from python to both api's, meaning that they should function with it; python is, itself, a language with a fairly large user base (bigger than C# and small than java), and it's dead simple to write and use. Very fast. And Clutter and Cairo look excellent; not necessarily like something that runs natively on any one system, but then, it's so fancy that it looks more next gen than simply different.
I'm hoping to use python and clutter, and if that doesn't work out, python and cairo.
lørdag den 28. marts 2009
Status on the generic storyline editor
My design needs to deal with the following:
Persistence
Syncrhonisation
Platform dependence
Programming Language dependence
Network dependence
Command response delay
User Interface
Backwards and forwards compatibility
Each area poses a unique challenge. Let's go over them one by one:
Persistence is necessary because an editor by definition needs to produce something persistent. But there are several kinds of persistence, and some are more valuable than others. One type is file serailization; this is a snapshot of the current results that is only readable by the current program and perhaps a hex editor. Another is XML-based serialisation; this allows for synchronization, limited interoperability with other programs, and limited readability by humans by means of standard text editors; but it's still limited to snapshots. The third, most advanced, and best option is persisting every change made in the editor along with a vector timestamp, an internet timestamp (if available), a local timestamp, and a user ID, and storing it as an atomic patch.
Synchronisation is necessary if more than one person uses the editor on the same project, or even if that one person works on the project from several locations. Backup is often considered an intrinsic part of syncrhonisation, because if you have access to synchronisation, you have access to backup.
There's a drastic difference between single-user and multi-user synchronisation; it's generally acceptable to use manual synchronisation between workstations, as well as manual backup, when there's only one user; further, it's also generally accepted that if that one user were to make different changes to his project at two different places without synchronising, all the changes made in one of the places would be lost upon synchronization: this limitation is accepted because a single user can generally "work around" this situation such that it never happens.
But for multi user synchronisation, it's generally necessary to allow simultaneous work in several places at once without having them manually reenter all the changes made in all but one place, in order to synchronize them.
Ideally, multiuser synchronization is either synchronous, or asynchronous but automatically run at set intervals and/or on demand, and when it occours, it usually does all, or almost all of the work without user intervention for solving conflicts.
Typically, synchronization also provides sync reports that go into detail on what was changed with each sync, so as to keep all users up to date. Synchronous synchronization is easy to implement, because everything is sync'ed all the time, but usually lags things down because all user clients need to be constantly chattering about whats happening all over the place, and demands solid network stability to the point of being always-on. Asynchronous sync is much harder to implement, but poses no strict network demands; instead, users must provide ID's, abd must help solve conflicts, but in return they're likely to get a snappy and free user experience.
Platform dependence is really OS dependence these days; all work done is done on PC's, so the only question is if the editor is able to run on all platforms natively, or if it has to happen on virtual machines. Platform independence is generally considered the more future-proof alternative, but it can pose some strict interface limitations because platform independent interface libraries generally look odd on most platforms.
Programming language dependence is closely associated to platform dependence; choosing a language will generally lock you in to using just that one language or having you incur rather tremendous costs, but there are different degrees of lockin. Python, for example, is often the second or third language that gets bindings to access libraries of another language, meaning you generally have several languages to fall back on. Some languages are also able to run on several different virtual machines, or interpreters, which can be a boon for future-proofing. Finally, some languages are exceedingly interoperable, allowing you to abandon them when a better one arises by means of automated translation.
Network dependence is just what it says; will the program need net access to function or not?
Command response delay is really a measure of how snappy the program is. If ever a user has to wait for the program, he becomes a _lot_ less productive; a short delay generally translates to a break in workflow, which makes a 10 second pause translate into several minutes of lost work. Some technologies _cost_ on this front; so it's one of those things that isn't an issue when you start out, but if you make the wrong tradeoff somewhere along the road, suddenly you have a useless program, or one with major drawback.
User Interface is just what it says; there are good user interfaces, there are clunky ones, there are bad ones, and there are ones that only a programmer can use, usually by command line. What type of user interface you need depend upon the target usergroup, and your ambitions with your application. The target audience of this program is fairly broad, since it simply means to get at creative individuals with no other qualifications, and they work better if they have a better interface. The ambitions are also high, of course, so that doesn't improve things. This makes the interface bit quite important.
Backwards and forwards compatibility is a matter of allowing the users to get at the program early while not hanging the sword of damocles over their head: If users are scared that future versions of the program won't be backwards compatible, they won't be inclined to use it.
So, now that we have some design parameters up, let me follow through with what I've been thinking, and how I've planned to tackle these issues so far...:
I've allowed the idea for a generic storyline editor to simmer for a while now, since about december 08 when I last did serious work on it, and have now come back to it.
And when I write idea, I mean plan. Traditionally, it's considered counterproductive to plan too far ahead concerning software development; rather, rapid prototyping and modular approaches tend to make for a far smoother creative process with far less grief over wasted effort.
It could be considered a mistake, then, that I've allowed my idea to linger for as long as I have. Thing is, allowing time to pass tends to reveal obvious design errors. One of which I feel I was close to making.
There is a reason, for example, that everybody hates the object-relational mismatch: The best way to store data is in databases, the best way to program is with objects. It's not just that these two methodologies are somewhat incompatible; it's that in spite of this incompatibility, using the two in conjunction is better than the alternatives for a wide range of software products.
Since a generic editor is by definition going to be a program with general utility, and since the projects meant to be edited by it are huge and may live to see several versions of it, such an editor closely mimics the utility pattern of enterprise software, and here's the point: It's precisely this type of software that usually needs both relational data storage and object based algorithms. Not because there aren't alternatives, but rather because the alternatives have been found wanting by the people making the decisions.
Fortunately, since the information needs to be available as objects in the application, I can start there, and simply be mindful that I only use plain serialization to disk if I need to persist anything at first. This means that the persistant data is, initially, bound to the programming language of the editor, but that's a small price to pay for utility before I have figured out how to write a database management module.
So, that means: persistence is going to be simple serialisation of objects for now, database stuff later, hopefully atomic patch-based.
Synchronisation will be solved by means of the database, again, later.
This means that, later, networking will be necessary for sync, but for nothing else.
This also means that there will be low command delay, since all the information will be available in local objects, always.
I've not made any final decisions on most of the other things, so that's the status for now.
Persistence
Syncrhonisation
Platform dependence
Programming Language dependence
Network dependence
Command response delay
User Interface
Backwards and forwards compatibility
Each area poses a unique challenge. Let's go over them one by one:
Persistence is necessary because an editor by definition needs to produce something persistent. But there are several kinds of persistence, and some are more valuable than others. One type is file serailization; this is a snapshot of the current results that is only readable by the current program and perhaps a hex editor. Another is XML-based serialisation; this allows for synchronization, limited interoperability with other programs, and limited readability by humans by means of standard text editors; but it's still limited to snapshots. The third, most advanced, and best option is persisting every change made in the editor along with a vector timestamp, an internet timestamp (if available), a local timestamp, and a user ID, and storing it as an atomic patch.
Synchronisation is necessary if more than one person uses the editor on the same project, or even if that one person works on the project from several locations. Backup is often considered an intrinsic part of syncrhonisation, because if you have access to synchronisation, you have access to backup.
There's a drastic difference between single-user and multi-user synchronisation; it's generally acceptable to use manual synchronisation between workstations, as well as manual backup, when there's only one user; further, it's also generally accepted that if that one user were to make different changes to his project at two different places without synchronising, all the changes made in one of the places would be lost upon synchronization: this limitation is accepted because a single user can generally "work around" this situation such that it never happens.
But for multi user synchronisation, it's generally necessary to allow simultaneous work in several places at once without having them manually reenter all the changes made in all but one place, in order to synchronize them.
Ideally, multiuser synchronization is either synchronous, or asynchronous but automatically run at set intervals and/or on demand, and when it occours, it usually does all, or almost all of the work without user intervention for solving conflicts.
Typically, synchronization also provides sync reports that go into detail on what was changed with each sync, so as to keep all users up to date. Synchronous synchronization is easy to implement, because everything is sync'ed all the time, but usually lags things down because all user clients need to be constantly chattering about whats happening all over the place, and demands solid network stability to the point of being always-on. Asynchronous sync is much harder to implement, but poses no strict network demands; instead, users must provide ID's, abd must help solve conflicts, but in return they're likely to get a snappy and free user experience.
Platform dependence is really OS dependence these days; all work done is done on PC's, so the only question is if the editor is able to run on all platforms natively, or if it has to happen on virtual machines. Platform independence is generally considered the more future-proof alternative, but it can pose some strict interface limitations because platform independent interface libraries generally look odd on most platforms.
Programming language dependence is closely associated to platform dependence; choosing a language will generally lock you in to using just that one language or having you incur rather tremendous costs, but there are different degrees of lockin. Python, for example, is often the second or third language that gets bindings to access libraries of another language, meaning you generally have several languages to fall back on. Some languages are also able to run on several different virtual machines, or interpreters, which can be a boon for future-proofing. Finally, some languages are exceedingly interoperable, allowing you to abandon them when a better one arises by means of automated translation.
Network dependence is just what it says; will the program need net access to function or not?
Command response delay is really a measure of how snappy the program is. If ever a user has to wait for the program, he becomes a _lot_ less productive; a short delay generally translates to a break in workflow, which makes a 10 second pause translate into several minutes of lost work. Some technologies _cost_ on this front; so it's one of those things that isn't an issue when you start out, but if you make the wrong tradeoff somewhere along the road, suddenly you have a useless program, or one with major drawback.
User Interface is just what it says; there are good user interfaces, there are clunky ones, there are bad ones, and there are ones that only a programmer can use, usually by command line. What type of user interface you need depend upon the target usergroup, and your ambitions with your application. The target audience of this program is fairly broad, since it simply means to get at creative individuals with no other qualifications, and they work better if they have a better interface. The ambitions are also high, of course, so that doesn't improve things. This makes the interface bit quite important.
Backwards and forwards compatibility is a matter of allowing the users to get at the program early while not hanging the sword of damocles over their head: If users are scared that future versions of the program won't be backwards compatible, they won't be inclined to use it.
So, now that we have some design parameters up, let me follow through with what I've been thinking, and how I've planned to tackle these issues so far...:
I've allowed the idea for a generic storyline editor to simmer for a while now, since about december 08 when I last did serious work on it, and have now come back to it.
And when I write idea, I mean plan. Traditionally, it's considered counterproductive to plan too far ahead concerning software development; rather, rapid prototyping and modular approaches tend to make for a far smoother creative process with far less grief over wasted effort.
It could be considered a mistake, then, that I've allowed my idea to linger for as long as I have. Thing is, allowing time to pass tends to reveal obvious design errors. One of which I feel I was close to making.
There is a reason, for example, that everybody hates the object-relational mismatch: The best way to store data is in databases, the best way to program is with objects. It's not just that these two methodologies are somewhat incompatible; it's that in spite of this incompatibility, using the two in conjunction is better than the alternatives for a wide range of software products.
Since a generic editor is by definition going to be a program with general utility, and since the projects meant to be edited by it are huge and may live to see several versions of it, such an editor closely mimics the utility pattern of enterprise software, and here's the point: It's precisely this type of software that usually needs both relational data storage and object based algorithms. Not because there aren't alternatives, but rather because the alternatives have been found wanting by the people making the decisions.
Fortunately, since the information needs to be available as objects in the application, I can start there, and simply be mindful that I only use plain serialization to disk if I need to persist anything at first. This means that the persistant data is, initially, bound to the programming language of the editor, but that's a small price to pay for utility before I have figured out how to write a database management module.
So, that means: persistence is going to be simple serialisation of objects for now, database stuff later, hopefully atomic patch-based.
Synchronisation will be solved by means of the database, again, later.
This means that, later, networking will be necessary for sync, but for nothing else.
This also means that there will be low command delay, since all the information will be available in local objects, always.
I've not made any final decisions on most of the other things, so that's the status for now.
Abonner på:
Opslag (Atom)