What should Apple do with Final Cut Pro?

I’ve had a couple of goes at writing this post, or a post like it. One approach had me riffing on what is most probably a quote from Henry Ford:

“If I’d asked people what they wanted, they would have asked for a faster horse.”

With time to consider, maybe that’s too forward looking, but my fondest hope is that Apple has taken the time to re-imagine Final Cut Pro and a NLE interface in general. Reality is that not every technology in the Apple arsenal could be added to Final Cut Pro and the other Pro Apps immediately.

Writing applications is in some ways similar to building a house. There’s usually an initial plan, often with stages to be added in future releases planned from the start. (I once asked a Digital Production BuZZ guest, around the time of FCP 4.5, if everything from their original wish list was in Final Cut Pro: response, nowhere near all!) However, over time, additions get added on; another “room” here, another one there. While you try and keep it all looking the same inevitably technologies change and the new bits don’t look and feel the same any more.

Ultimately it reaches a point where the original foundations, plumbing, electrical, sewer and other services can no longer keep up with the revised and rejigged building. At that time, the building is demolished and started over. In the physical world, the many rebuilt Las Vegas hotels would be a good example. In the NLE world, Premiere Pro would be one example: it’s all new code and modern code at that.

Without a doubt the latest NLE releases from Adobe and Avid have been very much “faster horses” – providing the features that people are asking for: faster, more real-time, native workflows etc. Meanwhile Apple remain their usual secretive selves and release a very not-revolutionary Final Cut Pro 7. Now there is a substantial amount of rewriting of the Final Cut Pro code, as I’ve written about before, but maybe that’s not all they’re doing?

Now for my standard disclaimer: I have no inside knowledge, no secrets leaked, no inside source. This is pure conjecture on my part, based on public data points.

When Final Cut Pro was released in 1999 it was new code, but since development had taken 3 years at Macromedia and nearly a further year at Apple before release, it wasn’t that new. Compared with Premiere 5/6 and Media Composer of the day, it was very much more modern code. Since then Premiere was rewritten to Premiere Pro (modern code) and Media Composer has been progressively rewritten over recent releases and we’ve seen the results in AMA and their open timeline.

Meanwhile we haven’t seen an enormous amount of progress in Final Cut Pro. Some nice new features, some definitely new code but it would appear Final Cut Pro is lagging the other major NLEs. We’ve seen “extensions” (to borrow my building metaphor) like Log and Transfer, Multicam, and other new features added since Final Cut Pro 4.5.

What if Apple – since they have to rewrite much of Final Cut Pro – decided to not just do a “faster horse” rewrite but rethink what the NLE should and could be? The first problem with making major improvements is that it will involve change and we know that no-one likes change: they want things to get better but never change! So if Apple are re-imagining Final Cut Pro, it will be unpopular with “the pros”, at least until they give it a try. (And I can probably name those who will hate it among my acquaintances.)

We know that Randy Ubilos – the original designer of Premiere 1 through to  4.2, Final Cut Pro, Aperture, iMovie 09, and iMovie for iPhone – is now Chief Architect of Video Applications, which I have conjectured to mean that he makes sure good video technology is shared among all the (secretive) divisions of Apple. They certainly need someone filling that role.

What “everyone” thinks “must” be in the next version of Final Cut Pro

Now, notice that i did not say “What Apple must do. I don’t run a company the size of the ProApps division, nor do I have inside insight on the decision making process. But that said, I think most people would expect to see Apple at least catching up to its competitors with:

• 64 bit native;
• 4 K and larger Timeline support;
• Native support for media that currently has to be transcoded or rewrapped into a QuickTime container;
• Use all the processor cores to their fullest;
• Better media and metadata management.

Ok, there probably aren’t that many people clamoring for better metadata management, but it’s a significant part of better media management, and crucially important for the future of automation in post production.

A 64 bit native Final Cut Studio would be able to work more reliably with large image sizes because it can address more memory: from the current limitation of 4 GB to virtually unlimited RAM access.

While Final Cut Pro can import images with more than 4,000 pixels in a dimension, a Sequence cannot be more than 4000 pixels in any one dimension. This is a very old, early QuickTime limitation that is part of the “old plumbing” of Final Cut Pro. 4K, for those who don’t keep the numbers in their head, is 4096 pixels wide. There are larger formats coming that people will want to edit in Final Cut Pro, such as RED Digital Cinema’s Epic camera.

While these are the most important things that Apple will need to address – to avoid having the user base complaining that Apple “don’t care” about professional video and have “abandoned” the Pro Apps – they are also the most difficult to address.

Adobe rewrote Premiere Pro leading up to an August 2003 release. Since this was modern code it was relatively easy to rewrite to 64 bit. The Mercury Engine rewrite took away dependencies on 32 bit code for media support (AVI and QuickTime) replacing it with nice new clean code. Adobe’s code is predominantly cross platform at the core, with a relatively small interface layer written in the native style for the platform. So Adobe quite reasonably claims “64 bit Cocoa”, which is completely true, but not what most people think. Neither Adobe nor Avid write against Apple’s Operating System Frameworks (APIs by a different name), whereas Apple does. Adobe writes the interface with Cocoa Frameworks (heavily subclassed to give the Adobe look, for those who care or even understand).

Avid have been progressively rewriting Media Composer, taking on a little of the rewrite with every release. Media Composer is neither claimed to be 64 bit (it’s still 32) nor does it use any Cocoa Interface Frameworks.

Apple have not been inactive. Pretty much everything written for Final Cut Pro since 4.5 has almost certainly be written in Cocoa. This would include Log and Transfer, HDV Log and Capture, presumably Multicam, Share, and Master Templates.

But the rest of Final Cut Pro 7 is still written with the older Carbon APIs that originally allowed an application to run on both OS 9 and OS X. This code has to be rewritten to Cocoa to achieve even these basic improvements and therein lies the problem.

QuickTime isn’t 64 bit. Well, parts of QuickTime are 64 bit in the QTKit framework. This work was done for QuickTime 7. However, as I’ve mentioned before, most of QuickTime has not been rewritten and Final Cut Pro calls many of these older APIs. You cannot build a 64 bit Final Cut Pro on 32 bit Carbon QuickTime APIs. Adobe seems to have worked around the problem by spinning off a 32 bit thread for QuickTime support so you lose that Mercury Engine goodness.

Apple could do that but there’s no way they could integrate it with Grand Central Dispatch (better use of all the available processor cores) or OpenCL (running code on the GPU), which would be key to performance increases and the use of all available processor cores. Intelligent use of Grand Central Dispatch and OpenCL would move Final Cut Pro’s performance more in line with Premiere Pro CS5’s Mercury Engine.

But no work appears to have been done on QTKit beyond what was needed to support QuickTime 7’s player and later the QuickTime X player. (Both call QTKit, which in turn will call the C APIs if there’s no Cocoa version available, although only the QuickTime 7 player will truly “fall through” to the C APIs). The QuickTime C APIs have over 550 “Classes” and more than 10,000 Methods. (Methods are commands in simple terms.) By comparison the 64 bit portion of QTKit contains just 24 classes and 360 methods! Now a lot of those Classes and Methods apply to features I expect to go from QuickTime: transitions, filters, interactive wired sprites, Flash support (already gone), and the rest. For more on the current state of QuickTime read this Ars Technica take on the state of QuickTime in Snow Leopard, particularly the references to Final Cut Pro at the end.)

There has been no apparent development of the QTKit framework for at least two years. What has been happening, as I posted in Introducing AV Foundation and the future of QuickTime, is a lot of work has been completed for media frameworks for iOS. AV Foundation in four years has 56 Classes and 460 methods.

That AV Foundation is to replace QTKit and the C APIs is a good thing because my reading of the Framework, Classes and Methods is that an AV Foundation based QuickTime would be able to support native media, something the current version cannot do. (Everything pretty much needs to be wrapped in a .MOV container.)

The first three bullet points above all require significant changes to QuickTime before they can be implemented. There are indications that the AV Foundation framework is slated to move to OS X desktop with OS X 10.7. This is not good news for the Pro Apps team, who would have liked it last year!

Now it’s possible that the Pro Apps Team have advanced builds of OS X 10.7 with the new AV Foundation-based QuickTime ahead of everyone else. This would be unusual, and it would be very unusual to build an app on top of such young code, except that it has been well tested in iOS. Even if they do have advanced access, it probably hasn’t happened much before the start of this year – or even the middle of this year – as AV Foundation’s real work was done for iOS 4 and 4.1. Real work, in this context, means the stuff we need for Final Cut Pro! It was only at 4.0 and 4.1 that the frameworks became “public”. They may have existed before that as private Classes as the two Classes introduced with 4.1 are almost certainly used for iMovie for iPhone released with iOS 4.0. It is generally not best practice to build on private frameworks (even if you’re inside Apple) because they frequently change before final release, but the urgency of rebuilding Final Cut Pro might mean they run that risk.

Even if they have advanced access to the new foundation, it’s going to be very tricky to deploy before OS X 10.7 is released. And we don’t have  date for that. Earliest expectation would be to announce at WWDC in June/July 11 and ship some time later that year.

To get all cores firing at full capacity would require linking to the Grand Central Dispatch framework, which is easy enough (relatively) once you have a Cocoa application. Similarly once you have a Cocoa application CoreData becomes available for use in media management.

The things I think Apple should probably think about doing.

Keep the media locations in project settings! Currently Scratch Disks are set as a General Preference with third party tools “switching” users. Rebuild it so media locations are project specific, wherever they are. And let us set our own locations for Capture, Render, Graphics but be aware of them in the project to allow direct browsing. Let us browse those “locations” with an interface similar to the way that iMovie and iDVD can access your iPhoto or iTunes library directly from within the interface.

Rebuild the Bin/Browser structure. That’s almost implied in upgrading metadata and media management support. Since Final Cut Pro 5.1.2 all metadata from non-tape sources has been stored as QuickTime Metadata within the Media. Not that we can access that from within Final Cut Pro (except by using miniME) but you have to believe there are plans to use it. Plus, the OS now has CoreData, which would be perfect to build a media database on. Final Cut Pro’s bin metadata display is essentially fixed, when it needs to be flexible so that any attribute-with-value that is in the file can be displayed as it can in Media Composer and Premiere Pro.

Make the Project format XML instead of Binary. Motion’s project format is XML, Premiere Pro’s project format is XML. There’s no reason why Final Cut Pro’s project format couldn’t be the XML format. The team did a lot of work on XML in Final Cut Pro 7 – something that’s not super obvious working with it, but is obvious when you’re deeply involved with Final Cut Pro XML.

Final Cut Pro XML has been one of Apple’s biggest success stories, becoming close to an industry standard. There’s no advantage to maintaining a separate binary (machine readable only) version of the project.

Fix all the bugs! I had to say it, even though it’s not realistic. Apple will almost certainly “fix” long standing bugs simply because they have been limitations in code that’s being rewritten. The reality of complex software development is that it’s impossible to ship code that is 100% bug free. Even internal QA and external beta testing can’t find every problem or imagine every circumstance that could occur. The next Final Cut Pro should be considered V1 code and will likely have some bugs.

Rethink Titling. I like that Randy Ubilos “though different” for Final Cut Pro’s titling needs, allowing multiple generators to be used in different way. This provided ultimate flexibility but frankly most people just want to compose a title over the actual video, and that’s not easy with the current tools. However, the OS now has some great text frameworks. Consider Pixelmator: while not as fully featured as Photoshop (mostly lacking selection and masking tools) it is built on OS X frameworks. It’s “just” (no offense, it’s work but not like writing from scratch) a front-end calling System frameworks.

Final Cut Pro could slide in a Core Animation layer over the current image in the Viewer with the Text (and paint) features right there, all thanks to code already written for Mac OS X.

Rethink the Interface. Reportedly Apple were looking to hire interface designers for the Pro Apps as recently as May 2010. I presume they’re hired by now, but you would expect a redesign to take at least a year to 18 months. (It is also possible that the positions were a smokescreen for other roles in the company. It happens but let’s assume face value here.)

Apple have some great interface elements in Cocoa, some of which were developed for ProApps. There are interface elements in iMovie 09 that looked interesting for Final Cut Pro (and certainly some who saw the LAFCPUG demo wanted access to those controls in FCP).

I’d love to be able to use QuickView on an item in the (new) Browser: press spacebar with the item selected and get a viewer, scroll down through items, just like in the Finder. There are a million interface designs possible with Core Animation. Would be great eye candy for the display to morph between Layouts. (I should add, that’s at the bottom of the interface overhaul list!).  Core Animation does open some nice interface elements.

The latest versions of the OS have been moving toward scalable interfaces, and I think Final Cut Pro would be a perfect example of the value of this technology. Right now the best we can do is choose text size, but a “written now” OS X application can be written so it scales with the display and scales intelligently so that borders remain proportional to the size the interface is being displayed, across all the various display sizes.

I’d love live search and auto-complete in the interface. The tools are there in Cocoa – we use them in prEdit – so once the Browser is rewritten, it should come standard.

And while we’re rethinking the interface, industry standard is definitely moving toward customizable interface colors. Now this one I’m not sure about. I suspect Apple’s attitude is along the lines of “we’ve given you the best possible color combination and you’ll only screw it up if we give you control”! Hopefully not. Even if it’s the Light to Dark slider Adobe provides in CS5. Avid are on the way to customizable colors with Media Composer but it’s still a “work in progress”.

Touch Interface. The more I use an iPhone and MacBook Pro with the touch interface, the more I love it. Working with Final Cut Pro now via touch feels like fun. Remember too that Motion has a whole Gesture library originally designed for pen and tablet input, but Touch is rapidly replacing that. I’m going to take fire for saying this, but I think Touch should be an important focus.

Background Rendering (to a cluster) Of course, with a good implementation of Grand Central Dispatch and OpenCL there’ll be a whole lot less rendering going on, but with file output increasingly more important than tape output (increasingly folks, you can’t argue with that) then rendering files becomes as important as real time performance, particularly as effects get more complex. Anything that needs rendering should just render in the background so it’s available.

AppleScriptable. When you’re writing a cross platform NLE you don’t focus on scriptable access specific to one platform. And that’s why Final Cut Pro has such poor (as in none) AppleScript support. Give us access to every command and menu item. Make the most common functions accessible to Automator. An AppleScriptable NLE, with a digital asset management server and I’m in workflow automation heaven.

More metadata automation. Well, part of me hopes they won’t because that’s my field, but it would be nice to see source metadata being used to auto-populate Titles or Master Templates (like iMovie for iPhone does).

Direct Access to Quartz Filters. We can get there now with tools like CHV’s but it would be nice to have direct access to filter building.

Better Integration with Final Cut Server. Let’s be able to call information from Final Cut Server within the Final Cut Pro interface, pull in media and have Final Cut Server handle all the media management to get it to the current project. For an interface comparison, think about how you can access your iPhoto or iTunes library in iMovie or iDVD.

Now I don’t expect that I’m even close, or they’ll be able to do everything in one release, but Apple, probably more than anyone else, have the right tools to really rethink the interface and leapfrog the competition with a whole new application.

I don’t even care if they keep Log and Capture or not!