Migrating from Stellent UCM & IBPM – A little foresight can alleviate a lot of trouble

Migrations from systems like IBPM to ILINX can be fraught with issues that can bite the unwary in very bad places. However, if you are aware of such problems, you can plan ways to mitigate them and have a successful migration in the end.

One issue we run into is documents that have a page or two with corrupt images. Perhaps when the page was first contributed to IBPM, a system or other type of issue caused the image to be corrupt or cease to exist. Either physical hardware or a software bug can be the culprit. The product we use for migration, ILINX Export, will flag this document as an error, skip it and move on to the next document in RECID order. Once the export is completed, these flagged documents have to be re-visited. Once a determination is made that an image is indeed corrupt, and the chance to recover it from backups is extremely remote, the document can be deleted or manually exported from IBPM without the corrupt image.

Another matter we’ve dealt with is related to non-tiff images. This category is “universal” type images, and includes PDF, DOC, XLS, MSG and a host of other file types that IBPM supports. There are options within the ILINX Export tool that will allow the export of these files types in their native format through the IBPM SDK. Or the export can be done through database manipulation that can directly access the image file and then “unzip” the universal file into its native format.

The issue that can be encountered here is twofold, and manifests itself when migrating to another repository. One, IBPM stores the native file zipped up with another file that contains metadata and has no file extension. When the document is unzipped there are two files, one with a valid file type and one without. Typically, backend repositories require file extensions, which are useful for performance, like displaying file type icon on the user interface, and a variety of other reasons. During the migration, importing to the backend may be impeded due to a lack of extensions on the metadata files. Secondly, if the extension of the universal file has been altered or damaged in storage, the file type may not be a standard that the new repository will accept. In any case, having your migration come to a screeching halt is something to avoid.

Awareness is the key. By proactively incorporating a response into your migration plan, you can eliminate much heartburn and anxiety. That is where the expertise and knowledge of a seasoned Optika / Stellent / Oracle integrator, like ImageSource, comes into play. We have helped many customers build migration plans that take these and other items into account, so the migrations are as smooth and worry-free as possible.

Oracle IPM 10g and Imaging 11g Migration: Part 2

A couple weeks ago I wrote a post about ECM migrations, with a focus specifically on moving content from Oracle IPM/Imaging to other destination systems—projects we’ve been performing a lot of lately. Our tools of choice for migrations are ILINX Export and ILINX Import, but if the destination ECM system isn’t supported by ILINX Import, there are other options. Almost every ECM system has mechanisms to do bulk or mass imports. ILINX Export provides many options to format the data so sometimes it is a matter of configuring the output to be in a format supported by the third party import application. Other times, utilizing these third party import applications may require a little development. Regardless of what’s necessary, we’ve never run into a destination system that we couldn’t work with.

There are multiple reasons we split the migration operations into two parts—export and import—flexibility being the biggest one. There are a lot more options when splitting the migration into two separate operations. Since we don’t modify the data on export from the source system, a snapshot can be taken for long term archival. Then on import, or pre-import, we can massage the data, perform file conversions, or augment the data by pulling additional data from an external source. Even though we split the migration up into two operations, they can be run in tandem so there is little effect on the overall duration of the migration.

One of the biggest concerns surrounding these migrations is the amount of time it will take. Performing tests in the actual environment is required because of how many variables go into the throughput of a migration. If the migration is estimated to take too long after initial testing, there are options to address that scenario, including:

  • Create a migration environment with instances of the source ECM system software on newer, more powerful servers, and restore the production data to these new servers in order to execute the migration from there. This has the additional benefit of removing any potential performance impact to the legacy production system for the duration of the migration.
  • Spin up additional instances of ILINX Export and/or ILNX Import to increase throughput. There will be a point when additional instances of the export or import process will not increase throughput—generally when a bottleneck restricts the maximum throughput that the source or destination system can achieve.

Recently, I had a customer that had set a hard go-live date that was just 60 days after project initiation for their new system. We had no problem meeting this requirement from a technology deployment standpoint, but our migration testing indicated that we wouldn’t be able to move all of their 25+ million documents in that time frame. In order to make the new system go-live date, we migrated the three previous years’ content first, then resumed with the older, remaining content. Since the vast majority of content to be retrieved would be from the previous year, the fact that the migration wasn’t 100% complete at go-live was a non-issue. This is an approach we’ve followed numerous times.

Once a migration is in full swing, auditing can be the most time consuming part of the process. ILINX Export and ILINX Import have very complete auditing capabilities, so while the migration is occurring, issues are immediately identified and can be addressed. We generally audit a couple different ways to confirm success. If only using ILINX Export, what is exported can be compared with what is in the source system to ensure all content was pulled out. When performing a complete migration, what is imported into the destination system is compared with the source system. Any migration can only be considered a success when it is proven that all the content was migrated, which is why we practice multi-step auditing during the migration.

By following our standard methodology for migrations and utilizing the technology we’ve developed over the years, we consistently perform reliably successful migrations. To read more about migrations, review my previous blog posts Oracle IPM 10g and Imaging 11g Migration and Steps for a successful ECM migration using ILINX Export.

If you have any questions about my blogs, or would like to discuss the possibilities for migration within your organization, please reach out to me or your contact at ImageSource to start the conversation.

John Linehan
Sr. Systems Engineer
ImageSource, Inc.

Transferring ILINX Release Configurations When Upgrading

Starting with ILINX Capture v6, the Release configurations are stored within the ILINX database. In ILINX Capture v5x, the ILINX Release configurations were stored in XML files on a disk. ILINX Capture called ILINX Release using a SendAndReceivedReply IXM. The change to store the settings within the ILINX database is very useful for a number of reasons: Release settings are part of the batch profile allowing for simpler migrations between environments, Release is much easier to configure, all configurations are in the database, etc. However, this change can create some extra work when upgrading from ILINX Capture 5x to ILINX Capture 6x. Because of the different architecture, ILINX Release needs to be completely reconfigured for the existing batch profiles. In addition, the Release XML doesn’t change, but there is a shortcut that can be taken. After you have upgraded ILINX Capture to v6, you’ll notice a new IXM in the palette: ILINX_Release_IXM_Icon

The existing ILINX workflow will likely have a SendAndReceiveReply IXM on the map that the 5x version of ILINX Capture used to call ILINX Release. Most likely, it would look like this:
SendAndReceiveReply_IXMTo configure ILINX Release for ILINX Capture 6x, the SendAndReceiveReply IXM will need to be removed from the map and a Release IXM must be dragged onto the workflow map in its place. Once the new Release IXM is on the map, it will need to be configured. This is where the shortcut can be taken. Instead of having to manually enter in the correct URLs, map the metadata values, and configure any other settings, do this:
Configure and save Release with some place holder settings: I normally leave the settings at default and enter in the bare minimum:

  • Job Name
  • User Name
  • Password
  • Batch Profile
  • Release Directory

Once ILINX Release configuration is saved and the workflow map is published, there will be a new entry in the ILINX Capture database Capture WorkflowAppSettings table. The CaptureWorkflowAppSettings.SettingsXML column is where the Release configuration is stored. Now it’s time to update the SettingsXML column with the XML from the ILINX Release 5x job settings file. The Release job should be on the ILINX Release 5.x server at c:\ProgramData\ImageSource\ILINX\Release\Settings\Jobs. The only caveat here is to be sure to place single quotes around the XML content. Here is what the SQL update statement would look like:

update [ILINX CAPTURE DATABASE].[dbo]. [CaptureWorkflowAppSettings]
set SettingsXml = ‘COPY AND PASTE ALL TEXT FROM 5.4 OR PRIOR RELEASE JOB SETTINGS FILE HERE’
where settingsID = ‘APPROIATE ID HERE’

Following this procedure can save some time if upgrading an ILINX Capture 5x system that has a lot of batch profiles. A lot of the time spent on the upgrade could be in the ILINX Release configuration. If I was upgrading a system with only a few batch profiles, I would probably just reconfigure them. If I was upgrading a system with a lot of batch profiles, I would go through the above steps to save some time.

John Linehan
Sr. Systems Engineer
ImageSource, Inc.

Failover Cluster Troubleshooting

There’s nothing quite like logging in to a customer’s system first thing Monday morning only to be greeted with this:

Cluster_report

I discovered this when I wasn’t able to log into the customer’s ILINX Capture implementation. The logged error (failure to locate the SQL Server) led me to take a look at the SQL Server’s configuration to confirm that its service was not running on either node of the cluster, and the error I got when trying to start that (a clustered resource could not be activated) led me to check on the clustered resources themselves.
Continue reading

Right way….wrong way

As with pretty much everything in life there is a right way to do something, and there is a wrong way to do something.  Unfortunately, it is often accompanied by that annoying “grey area” where it “sometimes” works.  This is an ever present problem when implementing and supporting hardware and software solutions.

Continue reading

Match Fields Early when Designing an Imaging Solution

When building an enterprise level imaging system, one of the most important early tasks is matching up the fields in the solution. A typical ERP imaging solution has fields in the following functional areas:

Scan and validation
Image repository
Workflow template
Temporary tables for line items and custom forms
EBS tables

Looking at the big picture, the entire imaging solution is simply a transportation system for meta data stored in table columns (fields) at various stages from paper scan to workflow, to voucher creation. Fields carry information that tie to a specific image document.

Many fields map across each step of the process and must have the same data types. Some fields are used uniquely during scanning, image retrieval, workflow, or in the ERP backend. It is very difficult to begin a project until all these fields are well-known and understood.

One of the challenges of matching up fields is that each area may refer to a field data type (string, integer, etc.) in a slightly different way. An experienced architect and solutions implementor will resolve the type quirks during implementation.

I’ve found the best way to understand how the fields map across the solution is to create a single table with a common set of fields names and data types with columns indicating areas of use, rather than creating separate tables spread throughout the project plan for each area.

Since project plans are lengthy, having a single field mapping table makes it much easier to create the necessary fields, templates, and tables during project implementation since you don’t have to jump around in the project document to find the needed information. Also, having a single table prevents incorrectly mapped fields between functional areas because the big picture view is simple to understand.

I like to use color to highlight specific logical field groupings and keep notes for each field. I tend to continuously revise the master table as the project unfolds and additional information is discovered and then email each revision to the key project players so that everyone is working from the same assumptions.

Here is a link to a sample field mapping table. The table can be improved by adding a column that contains realistic sample data for each field. I use basic field type nomenclature and convert the types as needed in each functional environment.

If you are responsible for putting together a project plan for an imaging system deployment, or work in a company with an imaging system, I encourage you to have a table like this that matches your system. It’s very useful.

Clint Lewis
Senior Systems Engineer
ImageSource, Inc.

 

 

Support: A day in the life of.

Recently Random and I were going over our daily activities, which can include internal support, Client support, and project work, and we were discussing how scattered and at times hectic our days can end up. We put together a list of events that could happen on any given day. This shows how techs covering several duties must be able to multitask and not pull your hair out in the process.

7:50 AM -Arrive at work, boot up laptop, and prepare for another work day.

7:55 AM – Check emails that came in over the night. It looks like we received a ticket regarding a client’s Oracle IPM that had crashed and they are not sure why. They attached the log information, but it will take some further investigation to fully vet the issue. Go to get coffee and found none ready so I needed to start a pot, I’ll come back later when the coffee is done.

8:00 AM – Begin work on an internal project that I was unable to work on much yesterday. Nothing scheduled for today so I should have time to work on it and I expect to finish it by the end of the day.

8:30 AM – Support Hours start.

8:35 AM – Reviewed Oracle IPM logs from the ticket that came in last night. I called the client and was only able to leave a voice mail.

8:55 AM – Returned to work on internal project.

9:02 AM – Internal Support: Employee calls asking for help with their second monitor. They cannot get it to come up even after several reboots. Continue reading