Engineering around Workflows

Here at The Digital Orchard, we develop complex software, while aiming to keep the user interface and experience simple, clean and predictable. That's our goal, and we believe that we've achieved that goal to a great degree. But we'll never be "finished" aiming for better.

Today, we investigated a situation where one of our clients, consisting of a small team of four people, was experiencing "failed saves" while using the custom application that we developed for them. This application manages their business' entire production workflow — inputing of information and content for the production of a weekly newsletter. We first started developing this application back in 2007, and it's received several new features and upgrades over the years.

One of the challenges of developing web-based applications is "statefulness"... the abilitiy for the application to maintain the state of what's displayed on-screen with what's stored in the back-end database. With multiple people accessing and editing records in this database, it becomes challenging to ensure that one person's saves don't interfere with another person's saves. Over time, we've continued to refine the technology that manages this collaborative editing, but new surprises are always around the corner. This is the very nature of complex and organic software.

The problem that we needed to solve this time involved foreground and background processing of certain database records. After an upgrade a few months ago, the software gained the ability to schedule a background update of a particular database record after it had been saved in the foreground (by the user). However, what we didn't anticipate was if the user chose to keep editing that record over several minutes:

1. The user would save changes to the record.
2. The application would schedule a background update of the record (for internal housekeeping reasons). This background update would be executed at the top of the next minute.
3. The user would save again, but if the background update had taken place by this point, then the "state" of the record that they had in the editor would conflict with what was in the database, and their save would fail.

After identifying the problems with the background processing happening while a user was actively editing a record, we made some changes to how the background update is scheduled. While the record editor is actively being used by someone, a call is made periodically to the server to bump the background update ahead by one minute. So that means that it will not run until the editor has been closed. Then the update will occur within one minute. This solution seems to work very well, and even works when mulitple people are editing the same record.

It feels great to solve a problem. That's probably our favorite part of the job, because afterall, software itself is designed to solve problems and to make life easier and more enjoyable.