Migrating from Stellent UCM & IBPM – A little foresight can alleviate a lot of trouble

Migrations from systems like IBPM to ILINX can be fraught with issues that can bite the unwary in very bad places. However, if you are aware of such problems, you can plan ways to mitigate them and have a successful migration in the end.

One issue we run into is documents that have a page or two with corrupt images. Perhaps when the page was first contributed to IBPM, a system or other type of issue caused the image to be corrupt or cease to exist. Either physical hardware or a software bug can be the culprit. The product we use for migration, ILINX Export, will flag this document as an error, skip it and move on to the next document in RECID order. Once the export is completed, these flagged documents have to be re-visited. Once a determination is made that an image is indeed corrupt, and the chance to recover it from backups is extremely remote, the document can be deleted or manually exported from IBPM without the corrupt image.

Another matter we’ve dealt with is related to non-tiff images. This category is “universal” type images, and includes PDF, DOC, XLS, MSG and a host of other file types that IBPM supports. There are options within the ILINX Export tool that will allow the export of these files types in their native format through the IBPM SDK. Or the export can be done through database manipulation that can directly access the image file and then “unzip” the universal file into its native format.

The issue that can be encountered here is twofold, and manifests itself when migrating to another repository. One, IBPM stores the native file zipped up with another file that contains metadata and has no file extension. When the document is unzipped there are two files, one with a valid file type and one without. Typically, backend repositories require file extensions, which are useful for performance, like displaying file type icon on the user interface, and a variety of other reasons. During the migration, importing to the backend may be impeded due to a lack of extensions on the metadata files. Secondly, if the extension of the universal file has been altered or damaged in storage, the file type may not be a standard that the new repository will accept. In any case, having your migration come to a screeching halt is something to avoid.

Awareness is the key. By proactively incorporating a response into your migration plan, you can eliminate much heartburn and anxiety. That is where the expertise and knowledge of a seasoned Optika / Stellent / Oracle integrator, like ImageSource, comes into play. We have helped many customers build migration plans that take these and other items into account, so the migrations are as smooth and worry-free as possible.

Oracle IPM 10g and Imaging 11g Migration: Part 2

A couple weeks ago I wrote a post about ECM migrations, with a focus specifically on moving content from Oracle IPM/Imaging to other destination systems—projects we’ve been performing a lot of lately. Our tools of choice for migrations are ILINX Export and ILINX Import, but if the destination ECM system isn’t supported by ILINX Import, there are other options. Almost every ECM system has mechanisms to do bulk or mass imports. ILINX Export provides many options to format the data so sometimes it is a matter of configuring the output to be in a format supported by the third party import application. Other times, utilizing these third party import applications may require a little development. Regardless of what’s necessary, we’ve never run into a destination system that we couldn’t work with.

There are multiple reasons we split the migration operations into two parts—export and import—flexibility being the biggest one. There are a lot more options when splitting the migration into two separate operations. Since we don’t modify the data on export from the source system, a snapshot can be taken for long term archival. Then on import, or pre-import, we can massage the data, perform file conversions, or augment the data by pulling additional data from an external source. Even though we split the migration up into two operations, they can be run in tandem so there is little effect on the overall duration of the migration.

One of the biggest concerns surrounding these migrations is the amount of time it will take. Performing tests in the actual environment is required because of how many variables go into the throughput of a migration. If the migration is estimated to take too long after initial testing, there are options to address that scenario, including:

  • Create a migration environment with instances of the source ECM system software on newer, more powerful servers, and restore the production data to these new servers in order to execute the migration from there. This has the additional benefit of removing any potential performance impact to the legacy production system for the duration of the migration.
  • Spin up additional instances of ILINX Export and/or ILNX Import to increase throughput. There will be a point when additional instances of the export or import process will not increase throughput—generally when a bottleneck restricts the maximum throughput that the source or destination system can achieve.

Recently, I had a customer that had set a hard go-live date that was just 60 days after project initiation for their new system. We had no problem meeting this requirement from a technology deployment standpoint, but our migration testing indicated that we wouldn’t be able to move all of their 25+ million documents in that time frame. In order to make the new system go-live date, we migrated the three previous years’ content first, then resumed with the older, remaining content. Since the vast majority of content to be retrieved would be from the previous year, the fact that the migration wasn’t 100% complete at go-live was a non-issue. This is an approach we’ve followed numerous times.

Once a migration is in full swing, auditing can be the most time consuming part of the process. ILINX Export and ILINX Import have very complete auditing capabilities, so while the migration is occurring, issues are immediately identified and can be addressed. We generally audit a couple different ways to confirm success. If only using ILINX Export, what is exported can be compared with what is in the source system to ensure all content was pulled out. When performing a complete migration, what is imported into the destination system is compared with the source system. Any migration can only be considered a success when it is proven that all the content was migrated, which is why we practice multi-step auditing during the migration.

By following our standard methodology for migrations and utilizing the technology we’ve developed over the years, we consistently perform reliably successful migrations. To read more about migrations, review my previous blog posts Oracle IPM 10g and Imaging 11g Migration and Steps for a successful ECM migration using ILINX Export.

If you have any questions about my blogs, or would like to discuss the possibilities for migration within your organization, please reach out to me or your contact at ImageSource to start the conversation.

John Linehan
Sr. Systems Engineer
ImageSource, Inc.

Transferring ILINX Release Configurations When Upgrading

Starting with ILINX Capture v6, the Release configurations are stored within the ILINX database. In ILINX Capture v5x, the ILINX Release configurations were stored in XML files on a disk. ILINX Capture called ILINX Release using a SendAndReceivedReply IXM. The change to store the settings within the ILINX database is very useful for a number of reasons: Release settings are part of the batch profile allowing for simpler migrations between environments, Release is much easier to configure, all configurations are in the database, etc. However, this change can create some extra work when upgrading from ILINX Capture 5x to ILINX Capture 6x. Because of the different architecture, ILINX Release needs to be completely reconfigured for the existing batch profiles. In addition, the Release XML doesn’t change, but there is a shortcut that can be taken. After you have upgraded ILINX Capture to v6, you’ll notice a new IXM in the palette: ILINX_Release_IXM_Icon

The existing ILINX workflow will likely have a SendAndReceiveReply IXM on the map that the 5x version of ILINX Capture used to call ILINX Release. Most likely, it would look like this:
SendAndReceiveReply_IXMTo configure ILINX Release for ILINX Capture 6x, the SendAndReceiveReply IXM will need to be removed from the map and a Release IXM must be dragged onto the workflow map in its place. Once the new Release IXM is on the map, it will need to be configured. This is where the shortcut can be taken. Instead of having to manually enter in the correct URLs, map the metadata values, and configure any other settings, do this:
Configure and save Release with some place holder settings: I normally leave the settings at default and enter in the bare minimum:

  • Job Name
  • User Name
  • Password
  • Batch Profile
  • Release Directory

Once ILINX Release configuration is saved and the workflow map is published, there will be a new entry in the ILINX Capture database Capture WorkflowAppSettings table. The CaptureWorkflowAppSettings.SettingsXML column is where the Release configuration is stored. Now it’s time to update the SettingsXML column with the XML from the ILINX Release 5x job settings file. The Release job should be on the ILINX Release 5.x server at c:\ProgramData\ImageSource\ILINX\Release\Settings\Jobs. The only caveat here is to be sure to place single quotes around the XML content. Here is what the SQL update statement would look like:

update [ILINX CAPTURE DATABASE].[dbo]. [CaptureWorkflowAppSettings]
set SettingsXml = ‘COPY AND PASTE ALL TEXT FROM 5.4 OR PRIOR RELEASE JOB SETTINGS FILE HERE’
where settingsID = ‘APPROIATE ID HERE’

Following this procedure can save some time if upgrading an ILINX Capture 5x system that has a lot of batch profiles. A lot of the time spent on the upgrade could be in the ILINX Release configuration. If I was upgrading a system with only a few batch profiles, I would probably just reconfigure them. If I was upgrading a system with a lot of batch profiles, I would go through the above steps to save some time.

John Linehan
Sr. Systems Engineer
ImageSource, Inc.

Failover Cluster Troubleshooting

There’s nothing quite like logging in to a customer’s system first thing Monday morning only to be greeted with this:

Cluster_report

I discovered this when I wasn’t able to log into the customer’s ILINX Capture implementation. The logged error (failure to locate the SQL Server) led me to take a look at the SQL Server’s configuration to confirm that its service was not running on either node of the cluster, and the error I got when trying to start that (a clustered resource could not be activated) led me to check on the clustered resources themselves.
Continue reading

Implementing SQL FILESTREAM Part II

Last month I wrote about enabling SQL FILESTREAM with ILINX Content Store. After discussing this with a few people, I think I should share some more information and reiterate a couple points.

For Existing Applications:
As I mentioned before, the decision to enable FILESTREAM should be done during the planning phase. If you perform this process on an application with a lot of content, it can be a very time costly endeavor with a big performance impact to the server. Also, after the move from BLOB to FILESTREAM, you could have a fragmented database. The BLOB to FILESTREAM process can definitely be done on an existing system, just be sure to plan accordingly and allow for sufficient time.

After step #10 of my previous blog post (all the data is copied and you have deleted the BLOB column), you will notice that the database file size hasn’t decreased. This is remedied easily enough be executing a DBCC CLEANTABLE command. The DBCC CLEANTABLE command will reclaim the space from the dropped variable length column. For example, if your database is named ILINX_CS and your application is named Sample Application, the query to do this is:

DBCC CLEANTABLE ('ILINX_CS','[dbo].[Sample Application]',10000) Continue reading

Registering DLLs in COM with WiX for creating an MSI installation package for a Kofax Custom Panel

I was working on a project recently for a customer that was upgrading their Kofax versions and making some enhancements to a custom Kofax panel that we had written for them some time ago. Like any good developer, I migrated the code for the custom panel to the latest version of Visual Studio I had, (in this case, Visual Studio 2012). I had finished development and was discussing installation when the customer requested an MSI package to install the custom panel. Unbeknownst to me, Visual Studio 2012 had dropped their support for the easy, drag and drop, built in set up and deployment project to create MSI’s.

In doing some research, I found many developers had migrated to using the open source WiX product to create MSI packages, (http://wix.codeplex.com/). One can download WiX and integrate it directly into Visual Studio. Everything was fairly straight forward on following their tutorials except for one snag: in order to get the custom Kofax panel to install correctly, I had to register the custom DLLs as COM Components, not in the GAC. After a lot of head scratching, I finally figured out that I could use Heat (one of the WiX tools) to create a registry file of the DLLs to include in my WiX set up project. You can find out more about Heat here: http://wixtoolset.org/documentation/manual/v3/overview/heat.html. After the file was generated I was able to take the output of the Heat generated file and include it in my WiX install project to register the necessary DLLs. To do this, I followed these steps: Continue reading

ILINX 6.X Database Lookup IXM

ILINX 6.X is an easy to configure and easy to use software package to scan, index, and provide workflow. The workflow steps are based on IXM (ILINX eXtension Modules) that are very similar to a programming language. There are several different types of IXM’s available out of the box. The following is a quick listing by name of the out of the box IXM’s:

5

By using the IXM’s, the designer of a workflow can have a batch move through single or multiple steps to perform any required task.

In addition to the IXM’s there can be actual code executed through a Client Side Extension or through a Server Side Extension. So there is little that cannot be accomplished using the ILINX Capture workflow IXM’s.

This week I would like to concentrate the discussion on a single IXM Database Lookup. The Database Lookup IXM is one of the most powerful when it comes to interacting with entities outside of ILINX. It not only allows ILINX to perform a database lookup and return column values to the Batch Profile or Document fields, but it also allows for the update of a database table’s columns. Continue reading