Prediction: I see the end of Application Automation

July 9 2008

How many times in your life can you say you have seen the demise of your best skill set and had a chance to react ... before that happened. I believe that situation is upon me, and I wanted to share some of my thoughts.

For the past 15 years or so, I have focused my technical skills on using one piece of software to automate another. Starting with Ami Pro macros that wrote data to an Excel spreadsheet, this has evolved. SmartSuite and Notes. Microsoft Office. OpenOffice.org. Lotus Symphony. Microsoft Project and Visio. And many more applications. I focused on direct automation (where one application controls another, like Notes using COM and OLE to generate a Word document) and indirect automation (where Word would call a web service to display and generate information, and that web service was from a Domino server). I saw this as a niche I could become the expert on. I believe I have done that and still find new and interesting things I can do for customers and pass along to folks at conferences and on this blog.

The problem with application automation is the following:

1. Application Automation has a high barrier of entry

The reality of why I speak on the topic of integrating Microsoft Office, OpenOffice.org, and Lotus Symphony with Lotus Notes and Domino at conferences every year is that this is not easy. After 15 years of doing this, I still find myself struggling with details of the process. This usually comes with high end functionality requirements or working with products like Project and Visio, but this really has not gotten that much easier over time. Sure, we moved from DDE to OLE to COM. We added web services which are standard. But the details are still way to complicated for the average developer. Frameworks like Integra for Notes has helped with this issue, but the top 25% of the development spectrum is still unreachable by 90% of the developers wanting to provide the functionality to their users.

2. Application Automation relies on many moving pieces

The second issue is that application automation requires many pieces. To have Notes automate a mail merge, we have the following moving pieces: Notes, Word, COM, and Windows. As much as Notes has been very upgrade friendly over the years, Word has not been. Mail Merge in Word has changed 4 times in the past 5 versions. Then you have COM and Windows. Application automation is 95% tied to the Windows platform. 5 years ago, that was no issue. Today, the explosion of Macs and Linux inside companies is requiring some changes. Do we really need to do a complete retesting of every piece of custom application development every time a new version of Office or Windows comes out? I believe the answer should be "NO" even if some companies will always do it. It should not be a requirement.

3. Application Automation creates lock in

Here is the key issue. If we building mail merge from Notes to Word, we have created a lock in situation. 99% of the time, if I want to change from Word to Lotus Symphony Documents, I have to rewrite the application automation from scratch. Sure, I can probably reuse the code that generates the address list in the case of mail merge, but too much of the work is thrown out. We need to move to writing solutions once that allow for user choice in the tools they user.

So where does this leave us? As I see it today, Application Automation is in the late Fall of it's life ... and Document Generation will take over.

Document Generation, as I define it, is creating a document from any application without using an editor. I am creating the document directly. We have done this for years with systems on mainframes and servers. Text files are created and processed. XML files are created and passed between services and servers via web services. I see this technique coming to the desktop application development world, specifically replacing Application Automation, over the next couple of years.

Imagine being able to create a document from Notes, presentation and data, without having to do it in Word. I can stop worrying about what the end user does while that happens. I can move the processing from the client to the server when possible. I can remove much of the complexity. I can create a file that numerous editors can open, both for read and edit. I can generate a read-only version automatically.

The formats of ODF and OOXML are going to bring us closer to this reality. Getting past the file format war that has taken place the past 18 months, standard document formats will release the developer. I would love to be able to generate a spreadsheet from Notes, on all three operating systems that Notes supports, readable and editable by any spreadsheet editor. I see that as a near future, not something far ahead of us. It will be better for everyone involved. The end user, the developer, the tester, and the administrator.

We have a long way to go before Document Generation will completely replace Application Automation. There is no standard for drawing or project formats yet. Most editors can not read and edit both ODF and OOXML today. And the general knowledge about both formats is still reserved for about 5% of the development community. All of these things will change in the next 12 to 24 months.

So where does this leave me? I will still be doing Application Automation for years, both for customers and as a speaker and author. But I am going to use the rest of 2008 to push myself to have as many Document Generation examples of all of my Application Automation demos and tools. My hope is to start to show them off on the blog over the next 6 months and highlight them on stage at Lotusphere 2009 (but submissions have not started, so that is far off and nothing is ever guaranteed). I also think the big push of XPages will really help with the Document Generation movement ... it's all just XML in the end. I also believe anyone with tools that provide any kind of Application Automation need to look at how to add Document Generation support. A good tool does not become obsolete with this shift, it just changes the way the file is created.

What a fun and exciting time. This shift is not something that scares me ... I am going to embrace it and be on the leading edge. Come along for the ride!