Data Engineer II (Library Systems)

Specific Duties & Responsibilities

• Collaborate with Business Analysts to manage roadmap for ETL and reporting projects.
• Serve as team lead for ETL and data exchange software development life cycle.
• Lead the development of tools and process to capture, normalize, and make available research and library collections data for study and research through the central data store and AI-ready data platform being developed by Library IT.
• Support library catalog and institutional repository systems through work with bibliographic metadata, including quality assurance, large-scale enrichment and corrections, and creating crosswalks to exchange or migrate metadata between systems that use different metadata schemas.
• Documentation of data pipelines for both developers and non-technical stakeholders following best practices for technical writing and systems diagrams.
• Contribute to the design, production, and maintenance of data pipelines for data acquisition, management, transformation, and back-end code development to power data web applications and convert raw data into usable information.
• Write and maintain ETL/ELTs that operate on a variety of structured and unstructured sources.
• Develop and maintain web data scraping systems for automatic data acquisition.
• Help design data architecture and provide ongoing support.
• Input/output data from databases and perform queries.
• Create scripts to clean, transform, and analyze data.
• Put into production data pipelines using data warehousing systems.
• Create and implement production software to monitor data quality and detect data anomalies.
• Perform daily manual data quality assurance tasks.
• Support, maintain, and troubleshoot the software infrastructure.
• Source data, conduct analyses, visualize data, and generate insights to support ongoing research projects and other requests across the organization.
• Collaborate with developers, analysts, data scientists, researchers, policy experts, and other partners.
• Communicate with division leadership, and others on the team.