The System Engineer Tool Belt and Problem Solving

Quite often, System Engineers are called upon to solve unusual problems. The greater our depth of knowledge about various technologies, the greater our problem solving ability.

My primary technology tool belt consists of knowledge in document capture, business workflow, enterprise content management, web development, .NET programming and web services, ERP Accounts Payable systems, and several other useful technology tools and systems.

Recently, a customer came to ImageSource with a problem. We had taught the customer how to use the PAWSER Oracle Imaging and Process Management (IPM) web SDK capability. What PAWSER makes possible is a specially constructed URL that can securely call an IPM system and retrieve a specific stored image. The P in PAWSER stands for “public” meaning there is no need for the calling user to log in to the imaging system. 

For example, consider the following URL:

http://mywebserver/IBPMExpress/External/DocumentActionProcessor.aspx?ToolName=PAWSER&SearchName=APSEARCH&CheckDate=11/18/2009
&CheckNumber=0&VendorNum=1234567&Viewer=ImagingPlugin

URL explanation:

“APSEARCH”: This is the name of the Saved Search being queried in Oracle IPM
“CheckDate”: The field being searched
“CheckNumber”: The field being searched
“VendorNum”: The field being searched
“Viewer=ImagingPlugin”: This launches the IPM viewer

In this case, the customer was dynamically constructing this url in a J.D. Edwards WORLD ERP system to retrieve an image related to an invoice screen. The problem is that the WORLD environment only allows 123 maximum characters for the URL string. The required URL for the call has more characters than that.

So stop here, don’t look below. How would you solve the URL length problem?

Fortunately, my tool belt consists of web programming knowledge. After a bit of thought, I realized it would be trivial to create a custom .NET ASPX page that could contain the fixed string data in the URL, retrieve the desired parameters in a much smaller URL from WORLD, then build the final URL with all the bits, and redirect to the actual PAWSER target page.

The following C#.NET code snippet resides in the custom ASPX page and shows one way this can be accomplished.

string target =
@"http://mywebserver/IBPMExpress/External/DocumentActionProcessor.aspx?
ToolName=PAWSER";
string searchname = "&SearchName=APSEARCH";
string cd = Request.QueryString["cd"]; //check date
string cn= Request.QueryString["cn"]; //check number
string vn= Request.QueryString["vn"]; //vendor number
string qs= "&CheckDate=" + cd + "&CheckNumber=" + cn + "&VendorNum="
+ vn; //QueryString
string viewer = "&Viewer=ImagingPlugin";
string url = target + searchname + qs+ viewer;
Response.Redirect(url);

The code can be refined to be more elegant and efficient using StringBuilder and array parsing in the QueryString, but the above example demonstrates the general approach.

If we name the custom ASPX page, apsearch.aspx, then the much smaller dynamic url from WORLD becomes:

http://mywebserver/apsearch.aspx?cd=11/18/2009&cn=0&vn=1234567

You may have come up with a completely different solution, but we can only use the technology tools we have in our tool belt.

Clint Lewis
Senior Systems Engineer
ImageSource, Inc.

  

Student Enrollment Transcript Processing

Many Colleges and Universities must handle transcripts received from other Colleges and Universities for Student Enrollment processing. The receiving College goes through several steps to process a Student Application for Enrollment and the associated transcript(s) that Application may have and those steps may require the application to pass to several different people.  Usually a folder is created to hold all the documents being received to support the Application and once all required documents are received this folder is passed on to Evaluators to evaluate the application and make an Admit or Deny decision.

 The processing of a transcript may follow different processing depending on the College.  In one case the information on the transcript is manually entered into an ERP system for Student Processing on a line by line basis.  This is very labor intensive and slows the processing of a Student’s Application.  In addition, the Evaluator must review the transcript and mark those lines that cannot be transferred, and manually add up the Units Attempted, Units Earned/Completed and calculate a GPA to see if the Student qualifies for admission.  A lot of manual processing is done on a single transcript and a single Student may have one or more transcripts from previous institutions and all have to be evaluated in the same way to determine admission to the school.

There are a couple of products available that can help to automate this Student Enrollment Processing, Oracle I/PM and ABBYY FlexiCapture.  Oracle I/PM can provide both the image storage of all the documents received and also a workflow to route the document sets through the various stages of processing electronically.  This relives the use of paper in the processing and the associated issues of losing track of documents or folders and time consuming searches to find a document.

 Since a paper transcript is different for different institutions a product that allows flexibility in processing different formats is required to read the data from the transcript and place it into similar fields that can be uploaded into an ERP system.  The ABBYY FlexiCapture product allows for the capture of information from a free format form like a transcript.  It has a module called FlexiLayout that allows the developer to specify where on a page a specific data set may reside.  It can handle table data like Session/Course data which can be repeated multiple times on a single transcript.  It can handle multiple page transcripts and multiple columns of data on a single page that continues on the next page.  This product is very flexible in the design stages to allow the developer to handle almost all the common issues when attempting to extract data from a transcript.

By using the ABBYY FlexiCapture product and releasing the extracted transcript data and the image into I/PM there are several time and labor gains to Enrollment Processing.

• Almost all manual routing of paper is eliminated.  This saves time in both the movement of folders from one desk to another and also saves time in searching for the correct folder to place newly acquired documents.
• Manual Line by Line data entry of transcripts is reduced.  Even with the ABBYY product some labor is still required to review the extraction results and ensure the data is correct.  However, this Validation step takes a lot less time and effort then manual line by line data entry.  The data can then be uploaded electronically into an ERP/Student Processing system.
• Since the extracted data is now in the I/PM repository it is easy to develop a form that can allow the evaluator to select the Session/Course lines to include in a Total Summary and then press a button so the totals are calculated automatically.  This sure beats the manual method of using a hand calculator.

 Using both of these products help in lowering the costs of processing Student Applications for Enrollment and the time consuming effort of transcript processing.

The Give and Take in Your Relationships

This week I thought I would take the blog post in a different direction.  Usually, we are focusing specifically on a technical topic and providing recommendations, solutions, theories and rants related to a topic.  I would like to take a step back from the granular technical details and talk about relationships.  Now I am not going to go and get all mushy, I want to focus on relationships in general that can be between any combination of entities and that will most likely apply to you.  This topic applies to any type of relationship whether it is between management and employees, a company and their customers/partners, coworkers, friends and family.  Enough prefacing, this week’s relations based topic is on the give and take aspect of our relationships.

Nobody likes to be involved in a relationship that is completely one sided.  “I perform, deliver, provide, pay for, etc… with no reciprocation from the other side.”  The purpose of this post is not to solve a specific problem or to get you to expect something in return for every little thing you do, but it is intended to get you to take a minute and evaluate the relationships in your life.  It is common for us to take advantage of good relationships or relationships that we have the upper hand in, just as it is common for us to complain and/or lash out in relationships where we are being taken advantage of.

Let’s look at some examples of one sided relationships in business:

• A customer spends money for a product or solution and the delivering company falls short in delivering the promised product, solution, or service.  The customer will feel cheated and the company will lose out in future business.
• A company continually delivers a quality product or successful solution to a customer, and an employee of the customer provides negative feedback regarding the company, solution, or service to attempt to reinforce their own value within their company.  This can be tricky because the customer is paying for the product or solution, but this creates a hostile or negative relationship which will limit future success for the customer.
• An employee continually performs at “above and beyond” levels and never receives appreciation from their company.  The employee will feel taken advantage of and there will most likely be a negative impact to the company from losing productivity or losing the productive employee.
• A company sells software for a vendor, and the vendor attempts to steal customers from the company.  The company will feel cheated by the vendor and do less or no business with the vendor in the future.

This list can go on and on, but by taking a look at a couple of real world examples we can see that there are winners and losers in each of these scenarios.  It is common for the immediate winners in these scenarios to become losers in the long run due to the resulting actions of the loser.  A lot of times the entity exploiting the relationship is not even aware that they are taking advantage of the other because of the immediate positive outcome of the relationship for them. 

Relationships are two way streets, and it doesn’t matter if they are business or personal, they require both parties to be committed to the relationship to make them successful and long lasting.  In my career and personal life I have found that actions will always speak louder than words.  By taking the time to appreciate and contribute to the relationships you are in, it will ensure success and prosperity in the end.  At ImageSource, we take pride in treating our customers as partners, having truth and integrity with our internal and external relationships, and the value of growing and strengthening our relationships on an ongoing basis.

Take a couple of minutes out of your day and evaluate the positive and negative relationships in your life.  For the positive ones, determine if you have a successful two way relationship or if you may be taking advantage of a good thing.  For the negative ones, determine if the other party is taking advantage of you, if it is self induced, or if it is due to a lack of communication.  Sometimes both parties can be at fault and everyone is a loser.  If you do find that you are being taken advantage of, determine how best to get the other side to understand, appreciate, and reciprocate accordingly.  In the end a successful two way relationship will create winners on both sides.

 -Ryan S. Keller

Making TeleForm and LiquidOffice Work Together

Scanning and capturing data via OCR can save lots of time over manual indexing.  Linking these same forms and metadata to an eForm workflow process takes it all to the next level of efficiency – processes that took hours and days can be reduced to minutes or seconds.  Using out-of-the-box connectors and settings, form templates that were created in Cardiff TeleForm can be exported to Cardiff LiquidOffice.  With a given form residing in both LiquidOffice and TeleForm, capture choices are broadened and the benefits of workflow are easily in reach.

Here’s how it is done.  TeleForm forms can be saved in a file exchange format that LiquidOffice Designer can read.  Once a TeleForm form is complete and tested, it is ready to go.  The form can be a traditional form that was created from scratch in TeleForm Designer, or it can be an ‘existing form’ created initially outside of TeleForm.  These forms then have TeleForm fields placed in appropriate data entry locations.

Once exported to the file exchange format, the file can be opened in LiquidOffice Designer.  It will look just like the TeleForm form.  An important step here is to save the file to the native LiquidOffice .XFM file format, then close LiquidOffice Designer.  Reopen LiquidOffice Designer and open the .XFM file – this quick step ensures the form can be previewed as an HTML or PDF form, or published to the LiquidOffice server, without error.  The fields in LiquidOffice retain important settings that were created in TeleForm such as field name, maximum length, and valid entry characters. If the form is a traditional TeleForm form, ID and reference marks can be optionally retained.  (This allows a LiquidOffice user to fill a form, print it, and fax or scan it in TeleForm.)

Now that we have identical forms in TeleForm and LiquidOffice, TeleForm can be configured to export metadata to LiquidOffice.   Open the TeleForm form in Designer and add the included LiquidOffice Export connect agent.  When configuring this connect agent, you supply the LiquidOffice form GUID or workflow process GUID.  The GUID uniquely identifies the form or process, and can be located via the LiquidOffice Management Console.   You also identify the LiquidOffice Virtual Submit Directory and routing information.  Since TeleForm can have many export agents attached to it, any current export may also be retained – they will coexist without complaint.   

When the form is processed in TeleForm – scanned, faxed, or even created using TeleForm Verifier’s NonForm Data Entry feature, the form will appear in the appropriate inbox or queue in LiquidOffice – filled and ready for approval, ad-hoc routing, or workflow processing.  Now that the form exists in LiquidOffice, you may also direct users to fill the forms directly from LiquidOffice when appropriate, bypassing TeleForm altogether.  The form now has grown wings and is ready to fly.

Integrating Disparate Applications at the Client Level

One product I really enjoy working with and thinking of ways to use is ILINX Integrate. The product is designed to bridge the gap between separate applications on the client side with no custom applications. If you can administer an ECM solution, configuring ILINX Integrate should come easily. ILINX Integrate is installed and configured at the client level so there is no need to modify the Line of Business or other external application. In a nutshell what the tool is designed for is passing data from one application to another. If you have implemented a server side integration in the past, think about it: how many hours were spent designing, implementing, and testing the integration? How many groups were involved and had to sign off on the idea of an integration before you could even start thinking about designing it. Personally I think that is the biggest benefit of an ILINX Integrate implementation, the ability to link two applications without having to make any changes to the applications you are integrating. From my experience the application owners don’t care, once they hear it is client side and no change will be made to the application they cease to be involved. Of course there is a flip side to this; every workstation you want to have the integration configured on will need to be touched. The product can be installed and configured remotely, but still there is a level of effort there.

It can be rather simple to decide whether ILINX integrate or a server side integration is a better fit. First off, can the applications you want to pass data between be modified or is there a built in mechanism for transferring data? If no, then a product like ILINX Integrate might be the only option. The other criteria for deciding if ILNX Integrate might be a fit is how many clients would need the proposed integration? If it is just a single work group or department that needs the integration, is it worth the level of effort to create the integration versus the cost of ILINX Integrate. The same applies to an enterprise implementation: compare the estimated level of effort for implementing a custom integration versus the estimated level of effort to distribute and manage ILINX Integrate to clients across the enterprise.

Examples of integrations ILINX Integrate can perform are:

• External database validations of data for applications that do not have the functionality built in. Any ODBC compliant database is accepted. For example, your ECM application does not have a mechanism for validating data integrity at input. With ILINX Integrate a data validation can be configured, a data validation can be configured by either adding a ‘floating’ button on the desktop or tying into an existing button on the screen.
• Transferring data between applications. Eliminate the need for Copy and Paste, configure ILINX Integrate to do that for you. I see this being deployed in a multitude of applications. For example, think about the last time you purchased a car. The dealership had to type in your data into a number of different systems: the lending system, manufacturer system, accounting system, etc. With an ILINX Integrate floating button they could type the data in once, click the button, and have it entered into all the disparate systems.
• Seamlessly integrate an ECM application (Oracle IPM, IBM FileNet, Microsoft SharePoint, etc) with your Line of Business application (JD Edwards, PeopleSoft, SAP, etc.). Right now users might be pulling up a record in the LOB application then manually navigating to their ECM application and performing a search. With ILINX Integrate that task can be performed automatically by tying into a button on the LOB page or screen, or adding the ‘floating’ button.
• There are many other tasks that ILINX Integrate can kick off automatically. Navigating to a dynamically built URL with data supplied from integrating application, there a number of file manipulation tasks, and decryption and encryption of data. There are many more tasks ILINX Integrate can be configured to perform as well.

I know I might have come off a little ‘salesy’ here but this is one of those products that achieves the main goal of virtually every project I have ever been involved with – save time and thus money. For more information about the product look at our product page for ILINX Integrate or contact us here for a demonstration.

Planning Applications and Repositories within Oracle IPM

One of the philosophies I have developed over the years is the separation of applications and their image repositories for every application developed. I build a storage volume and a storage class incorporating the volume for each individual application. These do not need to be physically distinct locations, they can be virtual names for the same location such as a Centera device or they can be two different folders on a SAN drive. The key is being able to reference them individually within the IPM product.

The advantage I have found is if there comes a time when the images must be separated, or different purge cycles are desired, these actions can be accomplished from the standard Oracle IPM interface. Also, the individual application could be spun off into its own installation with some work by a IPM professional.

The short term cost in time of building these extra storage class and volume references in my opinion is well worth the potential of not having limited your future actions with the system.

Jeff Doyle

Senior Systems Engineer

ImageSource, Inc.

ProStor and Oracle IPM — First Look

ProStor Systems sells a line of disk cartridge archival systems with some very compelling features. A representative of ProStor attended Nexus 2009 to demonstrate their systems, and as an Oracle IPM architect I was intrigued to see how well ProStor’s InfiniVault® would work in an IPM environment. So an associate and I visited ProStor’s headquarters in Boulder, Colorado with an Oracle IPM test system to put the InfiniVault system through its paces. 

We hooked up a direct network crossover cable to the archival system in the same NT Workgroup, and then attempted to configure IPM to talk to it. We found we had to set the IPM services account name and login to be exactly the same as configured in InfiniVault. Note that InfiniVault requires at least an 8 character password so the IPM services account must follow suit.

Once we had communication, images and universal documents flowed quickly into the archive system. Retrieval of objects from the archive system was very fast. We think setting IPM to archive older objects from expensive RAID 5 magnetic storage to InfiniVault can provide an opportunity to utilize the faster storage for current daily object retrieval, while placing less often accessed objects into long term storage. 

The ProStor system comes with a built in full text indexing feature we thought might be useful with IPM but unfortunately IPM stores all universal documents in a proprietary binary, with no file extension, which is what InfiniVault keys on to apply IFilters in order to index the data from many common file structures. 

InfiniVault also comes with a sophisticated records management capability but this can’t be used with IPM in any meaningful way due to the way IPM stores and tracks objects. However, the records management system could be used with many other common activities in an enterprise since InfiniVault can be used for all general archival tasks within an organization. 

Many more features are available with ProStor archival systems and we will be recommending them to our customers for IPM object archival. Feel free to contact me or  ImageSource for more information.

Clint Lewis
Senior Technical Architect
ImageSource, Inc.

Happy Thanksgiving

I would like to pass on a few fun facts about Thanksgiving.

Though it was not called Thanksgiving at the time, what we recognize as the first Thanksgiving feast was celebrated in 1621 by the pilgrims of the Plymouth colony along with about 90 Wampanoag Indians. The Pilgrims had suffered through a devastating winter in which nearly half their number died. Without the help of the Indians, all would have perished.

After the first harvest, Governor William Bradford proclaimed a day of thanksgiving and prayer to God. The food, which was eaten outdoors, included corn, geese, turkeys, ducks, eel, clams, leeks, plums, cod, bass, barley, venison and corn bread. The feast lasted 3 days. Though the exact date is unknown, the feast clearly took place in late autumn.

In 1623, a period of drought was answered by colonists with a proclamation of prayer and fasting. This prayer and fasting was changed to another thanksgiving celebration when rains came during the prayers. Later that year, Governor Bradford proclaimed November 29 as a time for pilgrims to gather and “listen to ye pastor and render thanksgiving to ye Almighty God for all His blessings.”

Throughout American history, there were many thanksgiving proclamations and celebrations. In 1789 George Washington proclaimed a National Thanksgiving Day on the last Thursday in November, in honor of the new United States Constitution. Thomas Jefferson, the third president, later discontinued it, calling it “a kingly practice.”

In 1863, Sarah Josepha Hale, the author of the poem “Mary Had a Little Lamb,” convinced Abraham Lincoln to proclaim Thanksgiving a national holiday. For the date she chose the last Thursday in November because of Washington’s proclamation. In 1941, it was officially changed to the fourth Thursday in November.

Since Abraham Lincoln’s proclamation, it has been a custom that all presidents of the United States make Thanksgiving proclamations every year.

ImageSource, Inc

  

Accounts Payable Processing with Kofax KTM and Oracle I/PM

Every company no matter how big or small has an Accounts Payable department.  It may be as small as one person or it may have ten, fifteen or more people.  For medium and large companies there are problems that arise with the data entry of all that invoice data and the tracking of all those invoices through the approval cycle.  There have been many different methods that companies have created to try and handle both which still lead to missing/late invoices and mistakes during data entry.  Methods such as logging the invoices either in a spreadsheet or manually on paper are just some of the types of methods that companies have tried to keep track of where invoices are and how long they have been waiting to be paid.  Maybe there is a better way.

 Kofax KTM in conjunction with Oracle I/PM provides a solution to these two issues.  Kofax provides two software packages to scan and then process documents like invoices.  KTM is the processing part and is able to OCR and identify the key fields on an invoice no matter where on the page the data is located.  KTM can also validate some of the data against the ERP system the company is running. Fields like PO Number, PO Release, Vendor Name, Vendor Address, Vendor Pay Site, Vendor Payment Terms and other fields can be read from the ERP system and added to the set of data held by KTM.  KTM can then pass this data on to Oracle I/PM.

 Oracle I/PM is an imaging and workflow software package.  Not only does it store the invoice document with the index data entered through KTM, but it also has a workflow module that can track the invoice through the business process so the users can know where in the process the document is sitting.  With I/PM workflow the invoice can be routed electronically to the assigned approver for invoice approval that is done on a Form built for workflow.  If there are multiple levels that can be handled within the workflow in one of several ways depending on the information available from other systems such as the company ERP system.  In addition, I/PM workflow scripts allow a programmer to write code to transfer the data directly into the ERP system through its data import tables.  I/PM has been successfully integrated to pass data to JDE F0411Z1/F0911Z1 tables and Oracle EBS Open Interface tables to name just a few.  These code scripts are re-usable so we can quickly get them working in your environment.  Have we have found, not every company process their AP Invoices in the same manner, so each implementation we do is slightly different than any other, so we need to discover the differences and then modify the form and script code to match.

 We at ImageSource can provide the expertise and experience to quickly get an imaging and workflow process into place in your company.  Just contact us at http://www.imagesourceinc.com to find out how.

The True Nature Of Beautiful Perfection

I’m always pleased when I can build a nice clean system for a customer.  I like to be able to look back and say, “That is beautiful.  I’m proud of that.”  Antoine de Saint-Exupery said, “A designer knows he has achieved perfection not when there is nothing left to add, but when there is nothing left to take away.”  I like that quote and try to build systems with that in mind.  I like to be able to simplify a customer’s business process so it makes more sense to them that it did before.

But there is a danger here that needs to be avoided.  We are not dealing with art or literature.  We are dealing with business systems, systems that receive input from untrusted sources.  This data needs to be checked.  Joel Splosky puts it best when describing things you should never do.  He talks about old code that “has grown little hairs and stuff on it and nobody knows why.”  He’s not describing something beautiful; he’s describing something that works.  Something that’s gone through the pain of having exceptions found and dealt with.

Exception Processing

We help customers automate their business.  The software products we sell all have a type of workflow build it.  Oracle IPM and Liquid Office have a true workflow component where you build your processes graphically.  ILINX Capture and Kofax Capture have the idea of queues and the ordering of queues.  Systems with combined software are generally designed to be used in stages such as scan, store and retrieve.  In each stage the data is moved from one queue to another or one piece of software to another and the data needs to arrive correctly.  These are the types of system interactions I’m focusing on.  Unfortunately, these systems aren’t always configured perfectly and something will happen.

Your scanned document will be unreadable.  Your form won’t be filled out completely.  You’ll have a power outage.  The database you rely on will have bad or missing data.  You network connection will drop.  You’ll need a strategy for handling these.

If you don’t handle exceptions well, your system will be consistently inconsistent.  Approvals will not be done on time.  Documents may be hard to find.  Users will become frustrated with extra unneeded steps to complete work.  Or worse, no one will notice until a year later and document XYZ is not in the system and the auditors that are in the room with you breathing down your neck lose their patience and your about to be sued.  You need to be able to handle these exceptions.

What An Exception Is Not

Just to be clear about the types of things I want to focus on, I’d like to list a few things I’m not talking about first.  When I say exception, I do not mean anything that can be considered a part of normal business processes.  For instance, in an accounting workflow, it might be part of the process to send invoices over $1,000,000 to a manager for special approval.  This is not an exception.  While the amount may be exceptional, the approval is a defined business process.  Conditional queues in your business process are also not exceptions.  If it is normal for a form to be missing specific data and there is a special queue for dealing with that, it is known and a normal part of the business processes.

What An Exception Is

Exceptions are when we are dealing with the unknown.  Exceptions are typically something that requires human intervention, special research or a new decision to be made that hasn’t been made before.  Exceptions are also when automated parts of the system suddenly stop doing the things they are suppose to.  (Actually, those are sometimes just programming bugs, but you need to handle those too.)

How To Handle Exceptions During Development & Testing

• Build maps and scripts that contain catch all routines. If you have a workflow script that needs to interface with an external system, make sure you can accurately report that the system is operational, that you can connect, that you can do the tasks you need to and that it will return successfully.  All of these steps may need their own error handling code so you can accurately report the problem.  You may also want to catch other exception outside the known ones so that you can log and report on them.  Workflow maps contain exception processing within them.  Most queues can return with an error, which can then be routed to a special queue.  Be willing to add those exception queues.  Allow users to be able to route their data to these exception queues with explanations of the issues.  They may not be used much, but you’ll be thankful there were there when something happens.

• Have good metrics and logging. The software we sell and implement already has logging features.  Those features need to be turned on.  When designing systems that have custom components, make sure they include a logging feature.  You will also need to make sure the data that is collected can be easily read.  This means investing some time into reviewing available reports or creating new reports.

• Include automatic reporting. Consider an automated way to flag problems or issues that can be detected.  Emailing a business administrator or system administrator is a good way to alert people of potential issues.  Be careful with this, however, as you can have too much.  You want your noise to signal ration to be low.  The last thing someone wants to do is to read through hundreds of automated emails because perhaps one indicates an issue.  (There is a reason most people turn off the Vista UAC.  Allow? Deny?)

• Test with real data. I’ve seen engineers send the same 5 documents through the test system thousands of times and declare the system is able to handle thousands of documents.  No, the system is able to handle those 5 documents.  That’s slightly different.  You need to test with thousands of different documents.  That is where your exceptions will be found.  You also need to test with real documents.  When some fills out a form for testing, of course they are going to fill it out correctly.  The programmer wants to make sure his code can read each and every field.  But what happens when they are not all filled in?  What happens when the data being used as a unique index is not there or not unique?  You need to test to find out.  Fake data will only test the common paths.  Real data will test for exceptions.

• Test with a lot of real data. I feel the need to state this twice.  Exceptions happen when users are using the system.  Use the system as much as possible with real data before you go live.

How To Handle Exceptions During Production

• Expect to find exceptions. Make it part of the production process to spend time looking at the log files and dealing with issues.  Every project our company manages includes what we call “rollout support.”  We know that exceptions are going to happen and we plan for the time to deal with and fix them.  Have someone whose job is to handle the exceptions from a business point of view, not just a IT administrator.  Give them the time required on a weekly basis to make sure the system is running smoothly, and that the users are happy with the way the system is running.

• Be willing to change your process. After an issue occurs a few times, it may make sense to consider it something that should be handled as normal processing.  Be willing to review the workflow map or order of queues.  Those conditional queues I mentioned earlier, the ones I said weren’t exceptions?  Those are really exceptions that everyone already knew about.  Now that you’ve found and know about one more, add it to the normal business process.

• Focus on the common problems. It’s okay to think, “This is the first and probably last time I will have to deal with this.”  As soon as the second time occurs, you need to plan to deal with more exceptions.

What To Take With You

By now you can see that your system is going to encounter exceptions too.  Your not alone.  We all have to deal with them.  You can see the importance of thinking about them, looking for them and dealing with them responsibly.  If you system is still being developed, you have the knowledge to be able to plan for them.  If you system is implemented, you now have good reasons to review and change it for the better.

I hope you can look at the design of your system and say, “The workflow map, scripts, queues and configuration may not be beautiful, but the system handles everything beautifully.”

Scott Hamilton
Senior Developer
ImageSource, Inc.

[photo by wikipedia user Patriarca12]