Designing and Managing Fisheries Data Systems that Support the NOAA Data Quality Act: A Case Study Using the Hawaii Longline Observer Program

Karen Sender
Pacific Islands Fisheries Science Center, NMFS

In summer 2001, cooperative work between data producers, data users and database administrators at the Pacific Islands Region and the Pacific Islands Fisheries Science Center identified problems with the existing Hawaii Longline Observer Data system. Poor data quality resulted from unsound database design and a general lack of policies and guidelines on how to manage data. The data system had far too many points-of-failure so that even a minor modification, such as adding an additional reference code to a data item, would result in a labor intensive effort to modify the data entry application, the database, and the processes to transfer the data to the regional database. Data summaries were reported from multiple sources, often with different results. Also identified was a lack of a clearly defined process for reporting and resolving data issues. Communication between data users and producers was problematic and, if a data issue was eventually resolved, it was likely to! reappear due to a lack of any system for process improvement.

In light of these issues, all data role-groups worked together to completely review and redesign the Longline Observer data system from data collection items and their definitions and collection methods, data collection forms, data entry application, data management practices and database design. Every effort was made to ensure data quality, transparency and utility.

During this process, the technical team at PIFSC made every effort to develop data management practices and tools that could be applied to other data systems. Presentation of the following applications will introduce these:

Longline Observer Data System - including user tracking and data auditing, reference code-management and an internet data entry application

Data Issues Manager - an internet application to report data issues and issue resolutions with electronic mail reporting of open and closed issues.

Data Validation Manager - system to develop, test and manage database validation statements within a database to ensure that all data are validated with a known and reproducible validation process.

Data Element Manager - internet application to document data items including information about their collection, management and accessibility.

Reference Codes Manager - internet application to manage and version data collection reference codes.