Government data is important to a municipality for the purpose of budgeting, forecasting, and creating public trust by illustrating transparency. The pillar of trust heavily relies on having reliable data that is both available to the public and the governing body for the municipality. Technology improves daily as vendors produce software that improves access to reports, graphs, and other necessary reports, graphs that contribute to the overall success of a municipality.
The keys to reporting data are quality, accuracy, availability, and reliability. Improperly configured software that allows for inaccurate data entry is a significant contributor in challenges to forecasting and reporting. A major consequence is rendering a decision based on the lack of factual data. The potential results in voting on legislation or fiscal budgets that is predicated on having accurate data is highly problematic and could present future challenges.
When deciding on a municipal software application, there are several factors to consider when going through the review process specifically pertaining to the data phase of the implementation. The number of software vendors grows annually and likely offer similar modules. The factors to consider in selecting the appropriate vendor suitable for your organization relies on deeper discussions to uncover potential hidden costs.
The importance of migrating data serves many purposes. For one, it allows for historical reporting to occur in a new system without interruption to continue analyzing data. Without the data being migrated, any analysis from the point of implementation moving forward has little benefit.
Secondly, the opportunity also exists to properly conduct a thorough review of the system and how historical data is viewed. In retrospect, data is data. Where the question arises is how each unique system interprets the data. A properly conducted migration will help expose deficiencies in the data and allow for correction in the new system.
And finally, the importance of migrating data allows for municipal audits to continue without delays. Audits are a key component in maintaining the financial health of a municipality. The presence of historical and current data to corroborate findings uncovered by the auditor helps to prevent any future reoccurrence of errors. The cleaner the data, the cleaner the audit and the results will improve the overall health of your municipality.
The 7 Data Migration Items
The parameters of the project require clarity, including a robust plan of deliverables and scheduling. Stakeholders are identified to indicate key personnel who will play a role in the data migration project. Clear lines of communication to establish points of contact will require extensive discussion in order to identify responsibilities along with a consistent mechanism to report on the data migration project.
Once the scoping is complete and the deliverables are met, the preparation portion of the project begins. Defining record layouts by the software vendor who will receive the data is a critical item during preparation. The record layouts for the current system will also help in the preparation phase in order to define data dictionaries that will assist in mapping the fields to the new system. Extraction of data and its method of delivery is key during this phase as security roles may require intervention from other departments. Cleansing scripts are created with preliminary loads of either test or a full complement of the datasets into a environment.
Validation of the data requires control totals and a preliminary load of the data during preparation for the purpose of validation the information. The critical piece of validation is that if the dataset prior to loading contained a certain number of records and yielded a specific amount of dollars, the control totals are used against the loaded data to validate accuracy. If those figures are unequal, a deeper dive into the scripts and resulting dataset is required.
Once the validation phase is completed and all data is verified, confirmation of the control totals subsequent to loading is the next step. The step is important and requires stakeholders to confirm the data prior to the preparation phase and after validation to ensure accuracy. Signing off on the resulting dataset allows for all parties to acknowledge the records are correct.
Standardizing is the process of reforming the confirmed dataset to the receiving software vendor. Most applications will gladly accept a comma separated values or fixed-width file. Others may accept Excel, dBase, SQL, and many commercially rated database systems. It is important to verify record layouts with the software vendor to ensure that the data is conforming to their specifics. There may also be a requirement for specific locations to upload the data, such as FTP. The details of delivering files is meant for discussion during the preparation phase and revisited during standardizing.
Once the final dataset has met all the standardizing requirements and placed in its final location, the import process begins. At this phase during the data migration project, control totals verified the process to be accurate. The deliverable discussed during the scoping phase indicates who is responsible for importing the standardized files. In most scenarios, reports are produced that indicate statistics of the final tables where the import process loaded the data. The number of records, matched fields, and dollars are merely an example of the control totals.
The most important phase is finalizing the data after the upload successfully completed. Testing in the new system to ensure accuracy is critical to completing the project. Working with the software vendor, it is incumbent to perform a complete test of all modules that contain the migrated data. during the finalizing phase, errors will be identified and corrected with stakeholders readily available to meet and coordinate any emergent solutions. conducting a complete audit of the project will also uncover any potential issues as users engage the new software product. A meeting of all stakeholders will allow for a discussion on project challenges, communication issues, and data anomalies may assist in the development of improved standards for the next data migration.
As discussed, data migration is an extremely important phase in any software implementation project. The steps to prepare data and load into a foreign system require a great deal amount of time, patience, and expertise. For larger municipalities, there may be departments who employ skilled individuals to embark on the project. For others, it may require consultants who specialize in assisting municipalities in migrating data from one software platform to another.