Creating a digital fingerprint archive for New York State law enforcement
This paper presents project management practices reflected in a successful imaging project, managing the creation of a critical digital fingerprint archive for New York State Division of Criminal Justice Services (DCJS). The expiration of a long-term warehouse lease provided a catalyst for implementing an electronic content management (ECM) digital archive for New York State's 22 million fingerprint records in Central Files, collocated in warehouse facilities. DCJS assessed the grave risk posed to these unique and critical law enforcement records, due to fire, water, or other damage. Loss of these records would seriously jeopardize public safety. The paper-based fingerprint records were expensive to maintain and manipulate, and poorly integrated with other critical DCJS systems and records. Creation of the digital archive achieved multiple high pay-off objectives in risk mitigation and cost avoidance.
Implementing the statewide automated fingerprint identification system (SAFIS) archive posed several challenges: a complex procurement process in a difficult economic environment; the sensitivity of the data; diverse technologies and architectures; and the need for extended contingency planning, when procurements were delayed, necessitating alternate site planning. This paper describes the lessons learned in managing the archive project, focusing on human resources, risk, and communications management, as well as project management for state government.
Criminal Justice in New York
State and local governments in New York State evidence multiple layers of public protection. At the state level, the Office of Homeland Security (OHS) holds responsibility for domestic security, while a consortium of law enforcement agencies are consolidated as New York State (NYS) Integrated Justice (IJ) -- Division of Criminal Justice Services (DCJS), New York State Police, Department of Correctional Services (DOCS), and Division of Parole. New York's counties employ sheriffs; cities, towns, and municipalities maintain police departments. New York's citizens are also served by district attorneys and district courts, at the state, county, city, town, and village levels.
Division of Criminal Justice Mission
DCJS enhances public safety and improves criminal justice through core functions (NYS DCJS, 2008):
- Criminal history checks and fingerprint operations;
- Timely information provided, public safety, law enforcement training;
- Breathalyzer and speed enforcement equipment repair;
- Accreditation of police departments and forensic laboratories; and
- Coordination of grant funds, Uniform Crime Reporting, research, hosting criminal justice boards and commissions.
Under executive direction, DCJS also holds the following additional priorities (NYS DCJS, 2008):
- Sex offender management – public outreach and education, community participation;
- Sentencing commission –top-to-bottom review of New York's sentencing statutes;
- Reduce violent and firearm crime;
- Reduce recidivism through re-entry;
- Increase DNA collection and compliance;
- Information systems and performance management; and
- Human trafficking.
Catalyst for Change
The expiration of a long-term warehouse lease provided a catalyst for implementing an electronic content management (ECM) digital archive for New York State's 22 million fingerprint records in Central Files, collocated in warehouse facilities. Criminal justice storage and retrieval requirements for fingerprint archives would add complexity, site re-engineering, scope, and significant cost to any new warehouse lease.
DCJS assessed the grave risk posed to these unique and critical law enforcement records, due to fire, water or other damage. Loss of these records would seriously jeopardize public safety. The paper-based fingerprint records were expensive to maintain and manipulate, and poorly integrated with other critical DCJS systems and records. Creation of the digital archive achieved multiple high pay-off objectives in risk mitigation and cost avoidance.
Implementing the state-wide automated fingerprint identification system (SAFIS) archive posed several challenges: a complex procurement process in a difficult economic environment; the sensitivity of the data; diverse technologies and architectures; and the need for extended contingency planning when procurements were delayed, necessitating alternate site planning.
New York State Fingerprint and Criminal History archive
New York State maintains one of the nation's largest collection of fingerprints, with over 20 million paper fingerprint cards and nearly 2 million other documents, stored in 2,150 filing cabinets and storage bins. These records were formerly housed in 17,500 square feet of storage in DCJS Central Files, with over 700,000 new cards added each year.
While DCJS undertakes critical tasks of fingerprint identification in its SAFIS system—soon to be replaced by the state-wide automated biometric identification system (SABIS)—fingerprint cards in the Central Files represent an official record of fingerprints for identification verification and court proceedings.
Until the creation of the digital archive, these records were manually maintained as paper records in numbered folders in over 2,000 file cabinets by 5 shifts spanning 24 hours a day nearly 7 days a week. Records submitted electronically were printed, and all paper documents stored by New York State Identification (NYSID) number. An average of 300 cards was requested daily for verification, court proceedings, and other identification processing, requiring manual retrieval from Central Files.
Risk Assessment and Cost Analysis
DCJS long recognized the huge vulnerabilities inherent in the unique and critical law enforcement records stored in Central Files. Through the years, localized leaks, spills, fire,s and other disruptions in the facility continued to underscore the risks to these vital records. Certainly the loss of these records would have jeopardized public safety, yet the maintenance and manipulation of these records came at an increasingly high cost in a time of severe budget challenges. The Central Files archives were completely isolated from other DCJS electronic systems and records, and in paper form, remained vulnerable to loss, with a high cost to maintain.
DCJS concluded that the creation of a digital fingerprint archive would achieve multiple high pay-off objectives, in terms of risk mitigation, cost avoidance, and benefits both tangible and intangible. Of paramount importance, the creation of a digital archive, with appropriate backup and data redundancy, would greatly reduce the risk of irretrievable loss of records essential to NYS law enforcement. Maintenance costs would also be significantly reduced, and digital records would offer greater processing efficiencies with increased integration with other DCJS systems and data. DCJS also anticipated a long-term increase in the quality of images through the use of digital image versus paper, due to the environmental effects on paper and ink over time.
Benefits of a Digital Archive
From a financial and efficiency standpoint, converting from a paper to digital archive was expected to achieve significant cost savings. DCJS would achieve considerable savings from the eventual elimination of printed card stock in Central Files. Fingerprints are captured on custom card stock at a higher cost than other paper records. DCJS also expected to reduce requisite support and management of the materials involved, such as reductions in printer maintenance and upgrade costs.
By migrating to a digital archive, DCJS would eliminate the need to print fingerprint cards from electronic files, merely to maintain these records in paper form along with other paper records. Over 90% of the fingerprint cards received by DCJS today are in an electronic message format (NIST) yet, after 90 days (or less, if the record is sealed before 90 days has elapsed) the electronic images were destroyed. Creating a digital archive would allow DCJS to reverse the then current practice of “keeping the paper and tossing the digital version.”
The reduction of personnel dedicated to maintenance of paper records in Central Files was estimated to be 11 fulltime equivalents (FTE), attributable to an increase in efficiency for the SAFIS Operations staff in the retrieval and use of archived documents, for a net US$310,000 savings. DCJS expected to eliminate approximately 85% of activities dedicated to management and maintenance of documents. DCJS would also achieve space savings (taxes, heat, A/C, and electric) of over US$235,000 annually. Given that hardcopy fingerprint records were often retained for early 100 years under the most stringent record retention requirements, DCJS will achieve decades of continued savings adding up to many millions of dollars.
Alignment with DCJS Strategy
Prior to implementing the digital archive, the DCJS computing environment lacked the ability to store a large number of images away from the main processing platform. Creating an archive large enough to hold NYS fingerprint images and related records, and allow appropriate integration with other systems and data, would require DCJS to construct a suitable archive platform within their environment.
DCJS concluded they would be better served by an enterprise solution for archiving, versus a targeted solution for just fingerprint records. This naturally led DCJS to consider the advantages of an enterprise content management (ECM) product, which would allow DCJS to store vast quantities of documents in multiple formats (e.g., JPEG, HTML, PDF, email, .doc), and linking them together. ECM software would also provide sophisticated search functions, an open architecture, and provide easier interface to other systems and processes.
By initiating the migration to true enterprise content management (ECM), DCJS achieved two DCJS executive priorities:
- Improving information systems and performance management by eliminating the practice of printing cards and implementing the archiving of electronically submitted fingerprint cards; and
- Supporting the conversion to SABIS by standardizing the current data stores for loading into the new databases.
DCJS operational users had serious concerns over image quality from scanned or converted documents. Documents could be used in court, requiring strict adherence to evidentiary requirements, chain of custody for evidence, and verification of procedures used to yield criminal identifications. While DCJS fingerprint examiners use the primary SAFIS for identification, corresponding records from the fingerprint archives constitute the evidentiary repository upon which formal identifications are supported.
New York's public safety depends on the accuracy of the identification system and the identification system depends heavily on the quality of the target database. Thus quality loss negatively affects public safety. To mitigate any possible loss of image quality, the DCJS SAFIS archive project confirmed image quality requirements with the scanning vendor, undertook extensive testing of scanning and printing configurations to ensure consistent quality of images. DCJS selected a specific computer, application, and printer combination that allowed for large scale, mass scanning with the highest quality image available using commercially available hardware and software. This was accomplished by using in-stock computer, server, and printer inventory items, with only a very modest expenditure on additional graphics software. DCJS also implemented proper quality assurance and testing to minimize image quality loss, undertaking a visual check on two percent of all scanned images.
Fixed completion deadlines for project phase components represented another serious risk to project success. The Phase 5 scanning needed to be completed prior to the expected Central Files facility closure date of February 28, 2009. The Phase 4 preparation of files ready for SABIS conversion initially needed to be completed by March 31, 2009. Initial mitigation included regular update of project progress and re-forecasting of Phase and activity deliverables throughout the project life cycle. Contingency plans relating to changes in scope and reassignment of resources were prepared to accommodate schedule delays. As it turned out, a combination of environmental changes, external project delays, and completed contingency plans allowed risks due to schedule and timeline to be avoided.
When scanning resources could not be hired in quantities necessary to recoup schedule delays in time for the expiration of the Central Files warehouse lease, multiple contingency plans were developed for securing alternate work locations for the file storage, preparation, scanning, and quality assurance (QA) operations, as well as extensive movement and logistics plans attendant to work allocation and distribution. At the end of the scanning effort, a final contingency plan allowed these same operations to be moved to existing DCJS facilities, allowing DCJS to engage the scanning vendor in post-project, additional scope scanning and knowledge transfer to in-house personnel.
Another perceived risk to the project was a slip in management and user sponsorship, due to perceptions that the scanning process would be intrusive and disruptive to ongoing operations. Plus, given the challenge of managing close to 14,000 boxes of fingerprint cards and other records, there was a risk that the project team would not be able to accurately track and report project progress to allow for good decision making on whether to invoke contingency plans.
The project team implemented a comprehensive system to track and monitor boxes of fingerprint cards at every stage of the storage, preparation, scanning, QA, and records retention processes. A dashboard was created, showing both counts of boxes in progress for each process, as well as cumulative project-to-date work completed. Each week, the project team facilitated a status review, to present dashboard results, identify issues, and discuss upcoming events.
Prior to the start of the project, the project team used various sampling and estimating techniques to determine the total number of records to be scanned, and the time required to prepare and scan fingerprint cards. Given the number of cards was in the 20-million range, DCJS decided to track work progress in terms of boxes of records, with boxes holding about 1,600 records, to allow more easier tracking and reporting. Boxes were tracked with unique box identifiers, as well as NYSID range annotations that allowed for easier card retrieval of records still inside preparation and scanning processes.
DCJS anticipated the risk of cost escalation due to miscalculation of the number of cards, or difficulties with cards. Some of the records were 50 years old or older. Throughout the project, the team continued to refine estimates using multiple, diverse sampling techniques. The project team monitored ongoing scanning progress, and compared actual versus forecast/estimates, and regularly developed revised forecasts for management.
As work progressed, project change requests were facilitated, in which additional resources were procured to prepare records properly for high speed scanning. Change requests were also initiated to allow for process changes that reduced storage requirements and allowed scanned records to be moved sooner to the State Records Center (SRC) for final storage until destruction. These added costs were offset by greater cost reductions due to extraneous paper and records not requiring retention, which were purged prior to scanning, and resulted in reduction in the number of records scanned.
Resource contention among other critical projects posed another significant project risk. Enterprise development and testing resources were required for coding and integration with existing systems, and elements of the ECM implementation needed to be coordinated with multiple technical units including facilities, network, infrastructure, storage area network (SAN), and database. There were also vendor personnel already on staff at DCJS for the existing fingerprint identification system, whose involvement in the ECM implementation might be advantageous, but for whom any reassignment to project tasks might pose a risk to vital maintenance of essential DCJS operations.
In addition to regular and frequent cross-functional status meetings and reporting, additional procurements were undertaken to augment internal technical resources with subject matter experts in ECM and imaging solutions. These expenditures had been anticipated and included in project spending estimated beginning with the initial business case, and specifically forecasted in the spending plan within the formal project scope document. Assignment of fingerprint identification vendor personnel was carefully controlled to limit involvement to staff not currently engaged in critical DCJS activities, or to additional resources contracted for specific work assignments.
Lastly, from a technical perspective, scanning equipment, software, and processes, as well as the introduction of an ECM, could have introduced incompatibilities in image and/or compression formats. Each component had to be evaluated against established standards and formats, and every aspect of the operation in its entirety had to be validated that the system would perform to specification, at the necessary levels of image resolution.
Other Project Management Challenges
Digital Archive and Data Sensitivity
During initial discussions about the potential for implementing a digital fingerprint archive, and throughout early project activities, project sponsors and stakeholders raised serious concerns about federal and state legal requirements, and whether DCJS would incur potential legal problems or judicial challenges related to either chain of custody or validity of fingerprint identifications. DCJS fingerprint examiners routinely testify in court about the basis for identifications and the processes used to acquire, compare, and verify identifications in criminal cases.
Conversion of public records from paper to other media began in the 1930s, and accelerated after World War II, primarily with many types of records converted to microfiche. These efforts were undertaken to protect records, reduce storage, and facilitate the distribution of information. In the 1980s, what had been converted to microfiche or microfilm were converted to digital media to increase the speed of processing and retrieval (Cohasset Associates, 2009).
By formal legal opinion, court finding, and explicitly in federal and state statutes, digital records have a legal standing equivalent to any paper version of the same records, given a demonstrable process and established practice for creating digital images. Formally, digital image copies have been validated as“writing” under federal rules of evidence. Just as paper records, digital image copies are admissible as evidence, provided such records can successfully pass long established evidentiary hurdles (Cohasset Associates, 2009).
Nevertheless, with the creation of the digital fingerprint archive, New York becomes the first state to convert their fingerprint records to a digital archive.
The Federal Bureau of Investigation (FBI) establishes fingerprint image quality specifications, security, and access standards and procedures to which NYS DCJS must adhere. To allow DCJS to implement contingency moves of fingerprint records to alternate sites for preparation, scanning, and QA, these procedures and specifications required thorough review. Enhanced security measures, including 24 hours a day, seven days a week, direct visual observation, door locks and access systems were implemented to maintain an acceptable level of security. Additionally, post-scanning retention of fingerprint cards adhered to guidelines established by the NY State Archives and Records Administration (SARA), and implemented as documents were placed in storage in SRC facilities.
Project start-up proved especially challenging. New York State Government provided a special procurement opportunity for work contracted through Industries for the Disabled, a nonprofit quasi-government entity whose mission is to foster and manage movernment procurements and services provided for people with disabilities. Many of the potential employees contracted under this arrangement frequently suffer from mental health and former substance abuse problems. These same characteristics often proved very challenging when potential scanning staff had to undergo site security screenings that included law enforcement background checks. Potential staff could not have serious prior problems with law enforcement to handle or have access to sensitive law enforcement records.
Compounding this problem was a relatively rigid security screening process ordinarily used for hiring permanent or long-term contractor staff. Designed for permanent or long-term employment, these processes were initially not very adaptable or responsive to the urgency of getting the scanning operation underway. Generally, facility preparations, hardware acquisitions, and site set up could be completed well in advance of the time needed to hire staff. This delayed full project ramp-up, and ultimately necessitated that the scanning effort be completed in a longer timeframe, with the implementation of contingency plans to move operations in advance of lease terminations.
The back end of the project experienced delays in hardware procurements prior to full scale ECM implementation. At project inception, the extensive testing of scanning and printing configurations led DCJS to adopt an image resolution higher than originally specified, which doubled the average size of scanned images. This required a later procurement of an additional 40 terabytes (Tb) of disk storage, adding to the already procured 40 Tb. This ultimately delayed the production load of scanned images to the new Oracle Universal Content Management (UCM) ECM system.
Lessons Learned and Results
What Worked Well
From project initiation, project management implemented a simple but effective method for scanning progress tracking and gained consensus from stakeholders to use these metrics. Since tracking work effort over the estimated 20 million records involved would have proven cumbersome, work effort was communicated in boxes of fingerprint cards, with an initial benchmark estimate of 1,600 cards per box. This number was continuously revised and updated weekly based on actual counts of cards per box. At the end of scanning, 12,900 boxes of fingerprint cards and others records were scanned, with an average of 1,515 records per box, for a total of 19.5 million records.
DCJS conducted a weekly project status meeting of project team and key stakeholders, reviewing a formal status report and a metrics dashboard, which graphically depicted the movement of over 14,000 boxes of records through initial inventory, storage, record preparation, scanning, QA, record disposal, and final storage. This dashboard, updated with metrics gathered from each component process, allowed senior management, stakeholders, and the project team to track progress and compare achieved results against forecasts. Regular re-forecasting as the project progressed also allowed stakeholders to properly plan for contingencies, and execute contingency plans effectively.
The DCJS Office of Criminal Justice Operations (Operations), responsible for maintaining the fingerprint archives in Central Files, engaged a Central Files manager with senior-level experience with fingerprint archive operations to oversee and manage day-to-day preparation of records for scanning. This manager selected and trained staff, gathered work metrics, responded to frequent changes and issues, enforced record preparation standards, and served as go-between for the digital archive project and the Central Files staff maintaining the paper files. This person accomplished all this with excellent results and incredible determination, all while managing normal Central Files management.
DCJS also engaged a scanning vendor with considerable experience with large scale scanning efforts within New York State Government. Scanning resources engaged through the Industry for the Disabled were enthusiastic, dedicated and professional. Between excellent management, and a professional and capable workforce, the scanning vendor demonstrated exceptional flexibility, responsiveness, and quality, at a very competitive and reasonable cost. By project end, the scanning vendor achieved an indexing accuracy of 99.996%, compared to a requirement of 99.95%. That resulted in an extra “sigma” of accuracy for these vital records.
Lastly, DCJS engaged a project sponsor with specific New York State, high volume, scanning subject matter expertise with another domain of critical state records. The project sponsor anticipated potential problems, conducted problem resolution and escalation with senior management, and greatly assisted the project manager, project team, and stakeholders in the successful completion of the project, and achieving success criteria.
Other Lessons Learned
The requirements for the digital fingerprint archive project were thoroughly defined in conjunction with a separate project initiative to upgrade the DCJS SAFIS System. These thorough and well-documented requirements greatly facilitated the identification of project work breakdown structure (WBS), appropriate assignment of resources during project execution, and anticipation of project requirements throughout the project life cycle.
The project sponsor was able to fully estimate total project costs based on a comprehensive set of requirements. With every facet of the project scope fully defined, most project costs were anticipated and accurately estimated. This “full scope” cost estimation allowed the project to be successfully completed within estimated costs. Conservative sampling metrics were used in scoping the number of records at an estimated 22 million records. Actual records scanned ended up at just over 19.5 million, over 10% lower than estimated. While reasonably accurate by any standard of records sampling, this relatively small over-estimation of the number of records to be scanned allowed the project to accommodate increased hardware costs of additional storage, and still yield a total cost lower than estimated.
DCJS senior management wholeheartedly committed to the digital archive project from project inception. Regular status reporting and the easy-to-follow dashboard allowed these decision makers to easily maintain project and situational awareness from inception to project completion. Their commitment to achieving project success greatly facilitated prompt problem resolution when any issue required management escalation.
Project management alerted stakeholders to the need for contingency planning as soon as schedule forecasts showed potential schedule slippage beyond the termination of the initial warehouse lease for Central Files. Contingency planning allowed DCJS to implement two successive moves of the entire storage, preparation, scanning and QA operations for the project. Critical stakeholders for the operations—and for the multiple disciplines and technical areas involved in the move—were drawn in to initial contingency planning, and kept involved through execution of those contingency plans. These efforts were highly successful and guaranteed a smoother project execution.
Association for Information and Image Management. (2008). Making a case for content management. Retrieved June 29, 2009, from http://www.aiim.org/Infonomics/ArticleView.aspx?id=34897
Cohasset Associates. (2008). The legality of digital image copies of paper records. Retrieved June 29, 2009, from http://www.aiim.org/PDFDocuments/36653.pdf
ID Networks. (2002). Electronic fingerprint archives for local and regional law enforcement. Retrieved June 29, 2009, from http://www.idnetworks.com/…/IDN_Whitepaper_FingerprintArchive.pdf
Keane, Inc. (2002) Productivity Management: Keane's Project Management Approach. Boston, MA: Keane, Inc.
New York State (NYS) Division of Criminal Justice Services (DCJS). (2008). About DCJS. Retrieved June 26, 2008, from http://www/internet/crimnet/about.htm
Project Management Institute. (2008). A guide to the project management body of knowledge (PMBOK® guide) (4th ed.). Newtown Square, PA: Project Management Institute.
University of the State of New York, State Education Department, State Archives and Records Administration (SARA). (2002). Retention and disposition of records: How long to keep records and how to destroy them. Retrieved June 29, 2009, from http://www.archives.nysed.gov/a/records/mr_pub41.pdf
University of the State of New York, State Education Department, State Archives and Records Administration (SARA). (2006) Imaging Production Guidelines. Retrieved on June 29, 2009 from http://www.archives.nysed.gov/a/records/mr_erecords_imgguides.pdf.
US Department of Justice (DOJ), Federal Bureau of Investigation (FBI). (2008) Integrated Automated Fingerprint Identification System or IAFIS. Retrieved on June 23, 2008 from http://www.fbi.gov/hq/cjisd/iafis.htm.
© 2009, Jeffrey C. Nuding
Originally published as a part of 2009 PMI Global Congress Proceedings – Orlando, Florida