Addteq Services recently had an opportunity to assist a biotechnology engineering firm in migrating defects from HP ALM to Jira. HP ALM software is designed to manage the various phases of the Software Development Life Cycle (SDLC), from requirements gathering to testing. Micro Focus acquired HP Quality Center/ALM at some point. However, this particular instance pre-dated that acquisition and was no longer supported. The customer had already migrated test cases to Zephyr Enterprise and wanted to migrate all defects to their Enterprise instance of Jira so that all teams could collaborate within a single tool.
Planning the Journey
The customer was able to extract defects and attachments on their own. All defects were exported from HP ALM to a CSV file in which each row represented a single defect, identified by a Defect ID. Attachments were extracted to a zipped archive, and a separate CSV file was created that defined the relationship between defect and attachment, with one row per attachment. Addteq needed this information to import the HP ALM defects into the customer’s Test Jira instance via CSV. Once imported, the customer would perform UAT, after which Addteq would migrate the content from the Test instance to Production. The original migration method involved Project Configurator, a tool the customer used for multiple project transfers. Sounds simple enough, right? Not everything is as simple as it seems. It soon became apparent that we had several challenges and hurdles to overcome.
Migration Challenges
The attachment CSV provided by the customer team contained one row per attachment and was formatted in the following way:
Defect ID | Summary | File Name | Path |
1 | Sample summary | agents.txt | ALM_Files\000\000\001 |
1 | Sample summary | image5.png | ALM_Files\000\000\002 |
1 | Sample summary | inventory.xlsx | ALM_Files\000\000\003 |
2 | This is a test | brochure.pdf | ALM_Files\000\000\004 |
3 | Another sample summary | presentation.pptx | ALM_Files\000\000\005 |
3 | Another sample summary | flowchart.vsdx | ALM_Files\000\000\006 |
If you’re familiar with file systems, you might have noticed that the slashes in the file paths conform to the Windows file system format. Since the customer’s Jira instances reside on Linux servers, the paths must be updated for Linux compatibility.
Although the customer extracted all attachments, the file paths provided pointed to generically named files; the files did not have the correct filenames or extensions, nor did the file paths in the CSV file. For example, in the path ALM_Files/000/000/001, the attachment’s file name is 001. It resided in the folder ALM_Files/000/000/ and needed to be renamed to ALM_Files/000/000/agents.txt. We immediately realized that:
- Each file needed to be renamed according to the names in the File Name column
- The path value for each attachment needed to be updated with the correct file name
While making these corrections, we realized that many file names included characters that were OK to use in the Windows file system but were not permitted in Linux, such as brackets and commas. Some defects referred to the same file name but referred to the attachment using inconsistent casing. (One might refer to attachment.txt while another referred to Attachment.txt)
Jira’s CSV importer requires admins to provide a single CSV file in which each row represents a single issue to be imported. To that end:
- The attachment file needed to be modified so that there was one row per defect rather than one row per attachment; if a defect had multiple attachments (as shown by Defect 1 in the example above), all attachment rows needed to be merged into a single row per defect with multiple attachment values per row, as needed.
- The attachment CSV needed to be merged with the primary defect export CSV to produce a single combined file for import.
And to top it off, the defect data included multiple date fields in different formats. Some columns were date only, and others were date and time. For example, examine the date fields below:
Defect ID | Review Date | Appointment Date |
4 | 04/01/2018 | 04/05/2018 09:30 AM |
5 | 12/31/2020 | 01/14/2021 01:00 PM |
6 | 06/24/2019 | 07/09/2019 10:45 AM |
Here you see that the Review Date field contains only dates, while the Appointment Date field contains a combination of dates and times. Due to a limitation in Jira’s external import facility, importing a mix of the two date formats is impossible:
Currently, the CSV file import wizard allows specifying a date format (this could be Date e.g “dd/MM/yyyy” or Datetime e.g “dd/MM/yyyy h:mm a“.
- If you specify the DateTime format: It will work for Datetime fields, but fail to import Date fields, and
- If you specify Date format, it will import both Date and DateTime fields but the DateTime fields will be stripped of the time digits. For instance, when “dd/MM/yyyy” is specified and you have a column with values like 25/12/2020 and another with 23/12/2020 9:15 PM.
- 25/12/2020 will be imported but,
- 23/12/2020 9:15 PM will be imported at 23/12/2020 00:00:00 (23/12/2020 12:00 AM)
Although Atlassian provided a workaround in their ticket, it was overly complicated, and the risk of error was high.
The Journey to Jira
Most challenges were solved by developing custom scripts that:
- Renamed the attachments
- Updated the file paths in the CSV
- Merged all attachment rows per defect so that there was only one row per defect, potentially with multiple attachments per defect
- Merged the defect data and attachment files into a unified CSV file that was ready for import
We devised an alternative to Atlassian’s workaround to ensure we could import all defects in a single import run. When importing data in date and time format to a purely date field, Jira strips out the time and retains only the date data. Therefore, all date data was converted to date time format using dummy time values to ensure all fields could be imported. This workaround allowed us to specify a date and time format for import, and Jira ignored the time values for date fields during the import process.
As with any incredible journey, there were a few bumps on the road. We had initially planned to use Project Configurator to export the project from the test instance and import it into the production instance. However, an exception was thrown while trying to import, which prevented the import from taking place. As there did not seem to be a simple or safe way to fix this issue quickly, we decided to use the CSV importer in the Production instance since the process had proven successful. Although Project Configurator could not import issues or attachments, it did import the project configuration, which created the project itself, and all objects, including custom fields and screens – saving us all the time and effort of creating these objects manually.
The Win
Immediately after migrating the defects to Jira, the customer started using their new Jira project and was able to schedule a decommissioning of their HP ALM instance for the very near future. The customer is delighted with the outcome and looks forward to working with the professional services team at Addteq for all their future administration and configuration needs.
We can help with all of your data migration needs.
