June 26, 2011, 5:19 a.m.
posted by un
Building a System: Babel Blaster
All the utilities in the book were designed so they could be used stand-alone as filters either by themselves or in combination with other filters. However, they were also designed so they could be built into a more comprehensive system. To develop that system the code from the examples has gone into an open source project being conducted under the GNU General Public License at http://babelblaster.sourceforge.net. As with any open source project run by volunteers, the requirements, what gets done, and when (or if) are all subject to who gets involved and how much time they can contribute. I'll outline here my proposals for the requirements for versions 1.0 and 1.1 and sketch out a few ideas about the overall architecture. If you want to find out the current status or help with the project, send me an e-mail ([email protected]).
Version 1.0 Requirements
I talked in Chapter 1 about functional and nonfunctional requirements. Let's approach Babel Blaster the same way.
We discussed enhancements in several chapters. Here is some of the functionality I propose for version 1.0.
Again, nonfunctional requirements of the system relate not to what the system does but to how it does it. These aspects deal with attributes of the system or its use such as cost, performance, and ease of use. For version 1.0 we'll keep the requirements I laid out in Chapter 1 such as maintainability and portability and add a few other important ones.
A nongoal for this version is performance improvements.
This seems to me a fairly comprehensive list. Getting all of these features into version 1.0 will depend on having enough volunteers involved. If you like to code this kind of thing, talk to me!
Even though I tried very hard to build these utilities using reusable modules, as we start looking at a larger architecture a few things may have to change. As any experienced designer knows, the shape of individual components can change as the overall system in which they are used changes. I'm sure that the lower-level classes will be okay, but the converter classes may need some adjustments. It is prudent to defer major architecture changes until after requirements have been finalized, so I'm not going to try to sketch out even a block diagram at this point. However, we can think about some general characteristics of the architecture.
To facilitate ease of use we can provide users with a single program to invoke. We can further simplify use by letting them specify the name of the input and a single identifier for the type of conversion to perform. This identifier would point to a set of data in a data store that would contain the file description document specification (or equivalent) and other relevant details. With EDI formats as the input, all that might be necessary is a generic identifier for the EDI syntax since EDI interchanges have a lot of identifying information in them.
The main program would retrieve the details from the data store about the first transformation step and execute the appropriate filter. Then, based on other details from the data store regarding succeeding transformation steps, it would invoke the appropriate filters and write the final output. Output location could be determined either by information retrieved from the data store or from a command line argument. Allowing a user-specified system-level command for either the initial or final step would provide a way to invoke file transmission utilities.
The overall architectural style remains pipe and filter, but the individual filters retain an object-oriented design. This means that the main program creates converter objects of the appropriate classes as it needs them.
Trading Partner/Application Information
In Chapter 9 we identified a need for a data store for EDI control information. In the current version of the XMLToX12 utility the data store was used only for generating sequential control numbers for interchanges and functional groups. This XML document could be expanded to include audit entries for interchanges and groups that are transmitted. It could also be used for tracking Functional Acknowledgments received. Because it would be an XML document, creating formatted reports, displaying it as a Web page, or developing simple queries against it would be fairly easy using standard tools.
In addition, we can observe that while the grammars of our various file formats are similar, there are some fundamental differences in processing CSV and flat files versus EDI. For EDI we can conceivably have one generic grammar that could cover all uses of an X12 transaction set or UN/EDIFACT message. However, for populating control segments in outbound interchanges a different set of information is required for each trading partner. It makes sense to set up a data store for EDI trading partner information and to break out the EDI grammars into separate documents. In addition, for ease of use we could set up entries in this data store, keyed off of short identifiers, that could be used to retrieve the file description documents for other legacy formats.
Linking Pipes and Filters
While we're still going to use pipes and filters for the general architectural style, we have at least two different choices for the types of pipe we use. The current utilities use the file system as a pipe. For simplicity, version 1.0 could continue to use the file system and simply fork child processes to execute stand-alone filters as necessary. However, all our conversion utilities are coded to deal with DOM documents. If we're passing them off to something else that can use a DOM document as input, there's no reason to write it out to the disk and read it back in again. This is no problem for the converters and presents only a minor problem for XSLT transformations. It is likely we'll want to set up direct calls to Xalan (Java or C++) from the main program rather than fork a child process.
Version 1.1 Requirements
I don't have many ideas at this point, but a couple come to mind.
We have avoided much emphasis on performance because we wanted to focus on maintainability, understandability, and functionality. However, I think if Babel Blaster gets to version 1.1 we might want to look at performance and ease of use.
As I mentioned earlier, if you'd like to volunteer your ideas and skills for Babel Blaster, feel free to contact me.