Jason C. McDonaldAs a card-carrying member of the Open Source Initiative, and an outspoken advocate for Linux, it shouldn't come as any surprise that I support open source software. At the same time, I get it: practically, not everything can be open source. We programmers have to make a living somehow, and open source doesn't always enable that.
However, no matter your software's goals, your business model, or your software license, there is one part of your project that ethically should be open source: your file format.
Cautionary Tales
The last 20 years of computer programming has proven that closed doors prevent innovation. Can you imagine the nasty mess we'd all be in right now if Netscape had never gone open source and evolved into Mozilla? The web standards we all take for granted would all be locked down by Microsoft et al, and the content creators, consumers, or both would have to pay the gatekeepers for the right to use the net!
Thankfully, Mozilla did happen, Microsoft lost its chokehold monopoly, and the phrase "open internet" means something.
Unfortunately, this same openness isn't ubiquitous, and the users suffer as a result. Consider the following examples:
End of Life
Adobe Flash is a once-closed file format that has only recently become partially open. Now that Adobe has decided to discontinue Flash, developers have to pay for the latest Adobe Animate CC and put in the work to update and recreate every project they ever made.
Huge pieces of the internet are going to stop working in a few short years, and most of them will not be replaced with their HTML5 equivalents. The open source Flash alternatives never stood a chance of catching up.
Regardless about how you feel about the medium, there's no denying that we're set to lose decades of content; all because Adobe locked down the Flash format.
In another fairly recent Adobe debacle, the FLV format was suddenly discontinued - as in entirely gutted out of its product line in an update without any warning. People lost years of work, and Adobe offered no recourse.
Problem #1: Other people's work is dependent on the continued existence of one entity. If the product or company ceases to exist (or be affordable, usable, etc.), that work is lost
Ransom Model
Many engineers and contractors rely on Autodesk's AutoCAD, partly because the DWG file format has become an industry standard. The file format is continually changed, to ensure only AutoCAD can open it, thereby forcing any professional with regular need to use CAD software to effectively pay what feels like a mob bribe for the right to work.
No matter how much Autodesk chooses to charge, users have to keep paying the subscription, or be unable to do their jobs. There are no options. Add in the Bus Factor - what happens if Autodesk ceases to exist? - and we've built an entire industry on one business. Remind you of anything? (See that infamous chapter in U.S. history called the Railroad Monopoly.)
Any time users are dependent on a closed-source file format, they are manipulated into paying an entity, lest they use their data. Hmmm...in a way, isn't that the definition of ransomware?
This introduces two problems:
Problem #2: It robs users of freedom. As soon as a user puts any work in using a closed-source file format, their project is bound to it for life. If the company lowers quality, abuses the user, or raises prices, the user has no recourse without throwing away their work.
Data Loss
The accounting world loves Intuit Quickbooks. I won't venture why - I've never used it, nor anything like it, so I won't pretend to evaluate its quality. The main complaint I have heard from many Quickbooks users is that their file format keeps changing, with little backwards compatibility.
If you happen to own one version of Quickbooks and you save your data, and then fail to upgrade every time, your data eventually becomes inaccessible. Later editions won't open the old files, and trying to may (from what I've been told) even somehow damage the software.
Problem #3: The controlling entity's decisions about the file format can cause data loss, and again, the users have no recourse.
Restricting Other Freedoms
If you're bound to any product that only runs on Windows, guess what operating system you are also bound to?
Yes, I am absolutely a Linux advocate, but let's set that aside for a moment. Do you realize that businesses stay with Windows XP or Windows 7 because they don't have a choice?
Obviously, open file formats don't necessarily prevent that entirely, but they do go a long way. For example, Microsoft Office users can move to Linux because both WPS Office and LibreOffice work well with DOCX files. However, AutoCAD users never can, no matter how good the alternatives get, since they're bound to DWG.
When you create a dependency on a closed file format for your users, you bind them to both your product and, in turn, everything your product is bound to. (Add in Problem #3, and that's a nasty little situation indeed!)
Problem #4: It prevents competition at multiple levels. Competition is the driving force that improves software. Given an ubiquitous closed file format, the software and its platform doesn't have to be good, or even tolerable. Superior products don't have a chance to compete against products with dominant file formats, and sometimes it isn't even seen as worth developing alternatives.
Devil's Advocate, Anyone?
Are there any honest pros to a closed-source file format? Let's look.
Security?
Many of us are celebrating the end of Adobe Flash because it is one of the most infamously insecure web platforms out there, and it has been since long before SWF was open-sourced. No, open source isn't inherently secure (looking at you, Node.js), but it is far more possible to catch vulnerabilities in open file formats, rather than closed ones.
Product Relevance?
I think many companies fear that, if they opened their file formats, then competitors would come along and take all their business. Maybe they would, but only if the competition was actually superior.
Customers like having actual choices. It is better to have customers who like your software for its features, interface, and support, rather than ones who are just paying ransom.
If you do wind up losing customers to a competitor because they outdid you, the battle is not over! I have tried products, liked them, but chosen something else for whatever reason. However, remembering the positive experience with the software I didn't choose sometimes led me back to them later.
Fair Competition?
One may argue, if competition is so important, why not compete in proprietary file formats, and let users decide? The issue is twofold:
Users can't decide once for all. Software changes, and so do project needs.
It is possible to change workflow mid-project, but it is not generally possible to change file formats.
Also, just because your file format is open does not mean competitors are going to immediately pick it up. For example, note that Audacity is the only program to save and open AUP project files, although both the program and the file format are open source. Audacity is one of the top go-to audio editing programs. No one has felt the need to replace them! Program and file format alike both won and hold their spot by sheer merit.
Software Reasons?
Maybe you don't want the user editing a file manually; perhaps a game savefile or some such. That's understandable, but a closed file format won't actually help you here. File formats are hacked all the time, game savefiles included. If you really don't want something viewed or edited, you should be using some standard form of strong encryption instead.
(Word of caution, of course: if your program is encrypting someone's project in a way that prevents the user from leaving your product, it's still ransomware.)
What's In It For Me?
I do believe it is safe to say that closed-source file formats have no honest advantages. By contrast, open-sourcing your file formats bring several advantages:
User confidence: In a market dominated by ransom schemes, conscientious users will have tangible reason to trust that you won't lock their project in a cage.
Competition: As weird as it sounds, this does help you! Competition drives us to create better software. If you win a large user base, you'll know it was because your software is presently superior.
Open Source Benefits: Even if your own software is proprietary, you may still benefit from the dynamics of open source via your open file format. The additions, fixes, and refinements others make (or suggest) to the file format benefit you, as well as anyone else using it.
Do It Right!
For a file format to be truly open, you must publish the standard. This shouldn't really be an additional step over the closed file format approach - if you're not actually documenting your file format somewhere, then you're asking for trouble as it is.
To truly benefit from an open standard yourself, you may additionally consider creating a formal process by which the file format can be improved.
Last Thoughts
I was a victim of Adobe's slow-death discontinuation of Adobe Flash. I know from an inside source that they had decided to kill off the Flash format years ago, but had kept it a secret from their users, even as they actively promoted the file format and sold people into it. In the process, my company lost $800, three years of work, and all trust in Adobe as a company. We moved to 100% open source file formats as a result.
Whether we're building open-source or closed-source software, we owe it to our users to take care with the trust they've put in us! We're there for them, and not the other way around. I believe this is a big part of that.
So build your software as open-source or proprietary as you see fit, but be sure you're not building the other type of ransomware. Open file formats are the only fair way to play.
