Tag:

data management

The critical role of HL7 and FHIR in Pharmaceutical Innovation: A Case Study Approach

by Massimo 2023-04-06

HL7 (Health Level Seven International) and FHIR (Fast Healthcare Interoperability Resources) are two critical standards in the healthcare industry, primarily focusing on the exchange, integration, sharing, and retrieval of electronic health information.

In the rapidly evolving landscape of healthcare and pharmaceuticals, the importance of efficient and secure data exchange cannot be overstated. Health Level Seven International (HL7) and Fast Healthcare Interoperability Resources (FHIR) have emerged as pivotal standards in this domain, revolutionizing how healthcare data is shared and utilized. For pharmaceutical companies, these standards are not just technical guidelines but essential tools that drive innovation, enhance patient care, and streamline operations.

The importance of HL7 and FHIR

HL7 and FHIR standards play a crucial role in the pharmaceutical industry. They enable seamless communication between disparate healthcare systems, ensuring that critical patient data is accessible and interpretable across platforms.
This interoperability is vital for pharmaceutical companies in several ways:

Enhanced Drug Development: Access to comprehensive and standardized patient data can significantly accelerate the drug development process. HL7 and FHIR facilitate this by ensuring data from clinical trials and research is easily shareable and analyzable.
Improved Patient Outcomes: With standardized data exchange, pharmaceutical companies can more effectively track drug efficacy and patient outcomes, leading to improved medication and therapy plans.
Regulatory Compliance: Adhering to HL7 and FHIR standards helps pharmaceutical companies comply with stringent data handling regulations, reducing legal risks and ensuring patient privacy.

Technical example

An example of a data standard in the form of a JSON object is posted below. This example could represent a simplified patient record in a healthcare system, following a standardized format that might be used in systems adhering to FHIR or similar data exchange standards.

This JSON object includes various fields such as patient identification, contact details, medical history, and recent visits, all organized in a structured and easily readable format. This standardization of data enables efficient sharing and processing across different healthcare systems and applications.

Patient Information

ID: 123456

Name: John Doe

Date of Birth: 1980-01-01

Gender: male

Phone: +123456789

Email: johndoe@example.com

Address: 123 Main St, Anytown, Anystate, 12345

Medical History

Allergies: Penicillin, Aspirin

Chronic Conditions: Diabetes

Medications:

Metformin (500mg, twice a day)

Recent Visits

Date: 2023-03-15
Reason: Routine check-up
Physician: Dr. Smith
Notes: General good health
Date: 2023-02-10
Reason: Flu symptoms
Physician: Dr. Jones
Notes: Prescribed flu medication

Case Study: integration of FHIR in a pharma company

A real-world example of the impact of these standards can be seen in the case of I worked on in the past for a global R&D company that integrated FHIR standards into its data management systems with the following results:

Streamlined clinical trials: By adopting FHIR, were were able to integrate data from various clinical trial sites seamlessly. This integration led to a 30% reduction in the time required to collate and analyze clinical trial data, expediting the drug development process.
Enhanced data security and compliance: FHIR’s robust framework ensured that patient data collected during trials was securely managed and compliant with global data privacy regulations, such as GDPR and HIPAA.
Improved Collaboration with Healthcare Providers: FHIR-enabled systems allowed to share trial results and drug information efficiently with healthcare providers, ensuring that patients received the most up-to-date and effective treatments.

This case illustrates how the adoption of FHIR facilitated improvements in data management and analysis, directly impacting research and development (R&D) efficiency and clinical trial processes.

Implementation Phases

1. Assessment and Planning

– Gap Analysis: we conducted a thorough gap analysis to understand the discrepancies between their current data systems and the requirements of FHIR.
– Strategic Planning: we developed a strategic plan focusing on infrastructure upgrades, software development, and training needs to align with FHIR standards.

2. Infrastructure Overhaul

– Upgrading Data Repositories: we upgraded data storage and processing capabilities to handle FHIR-based data structures. I will not go too much in details of this step because it is quite depending on the infrastructure a Company might have already deployed.
– Network Security: Given the sensitivity of clinical trial data, our IT enhanced network security to comply with FHIR’s strict data security protocols.

3. Software Development

– FHIR-Compatible Tools: We developed and integrated FHIR-compatible software tool from Cerner for data capture, storage, and analysis.
– API Integration: APIs conforming to FHIR standards were implemented to ensure seamless data exchange between internal systems and external entities like trial sites.

4. Data Migration and Validation

– Data Conversion: Existing data was converted into FHIR-compliant formats.
– Quality Assurance: Rigorous testing was conducted to ensure the integrity and accuracy of the migrated data. Quality Assurance (QA) is indeed a critical component in the data migration process, ensuring the integrity and accuracy of the data being transferred to the new FHIR-compliant system. In our Quality Assurance phase, we conducted rigorous testing to ensure the integrity and accuracy of the data migrated to the FHIR-compliant format as listed below. This thorough validation process was crucial in confirming that the data not only conformed to the new standards but also maintained its reliability, a key factor in supporting our subsequent R&D and clinical trial activities.

Data Validation Test (Ensuring the migrated data matches the original source data in terms of content and format)
Schema Compliance Test (Verifying that the data structure adheres to the FHIR schema and standards)
Integrity Test (Checking for data completeness and accuracy, ensuring no corruption occurred during migration)
Functionality Test (Assessing if the data can be successfully retrieved, updated, and deleted in the new system)
Performance Test (Evaluating the system’s response time and speed in handling FHIR-compliant data)
Security Test (Ensuring that data is protected and complies with privacy regulations like HIPAA)
Interoperability Test (Confirming that the migrated data can interact effectively with other FHIR-compliant systems)
User Acceptance Test (Validating the system with end-users to ensure it meets their needs and expectations)

Practical Activities in R&D Analytics

Data Standardization in R&D: we research data, including genomic, clinical, and drug efficacy data, into FHIR formats, facilitating easier aggregation and analysis.
Advanced Analytics Implementation: The use of FHIR standards allowed for the implementation of advanced analytics and machine learning models to predict drug responses and identify potential new drug targets.

Clinical Trial Enhancements

Real-time Data Access: The adoption of FHIR enabled real-time access to patient data from clinical trial sites, allowing immediate analysis and decision-making.
Enhanced Patient Recruitment: FHIR’s interoperability features allowed for more efficient patient recruitment by matching patient profiles with trial requirements across different systems.
Remote Monitoring and Reporting: the implementation of FHIR facilitated remote monitoring of trial participants and automated reporting of trial results, significantly reducing manual data entry and potential errors.
Regulatory Compliance: clinical trial data handling was in compliance with global standards, facilitating smoother regulatory approval processes.

Data management in practice

by Massimo 2023-03-14

My experience in data management

In this post, I want to share the challenges I experienced in the context of data management, summarizing some of the activities I had to deal with, working with matrixed organizations. Let me re-order the activities in function of the data life cycle and the framework for data management.

As a data manager, one of the first milestones I had was designing and implementing a new data management strategy that outlined how data was to be collected, stored, processed, analyzed, and reported. This strategy ensured that the organization’s data was accurate, timely, and accessible, and that it complied with relevant laws and regulations.

To ensure that the organization’s data was reliable and accurate, we developed a data quality framework. This framework defined the criteria for data accuracy, completeness, consistency, and validity, and established procedures for monitoring and addressing data quality issues.

Then we created data visualization tools that helped stakeholders better understand complex data sets and identify patterns and trends. These tools included user-friendly dashboards and reports that presented data in a visually appealing and easy-to-understand way.

Data sharing protocols were critical for organizations that collaborated with other entities, such as government agencies or NGOs. I established new protocols for data sharing that protected the privacy and security of sensitive information while ensuring that relevant stakeholders had access to the data they needed.

Data governance policies established rules and procedures for data management, including data ownership, access, and use: we implemented data governance policies that aligned with the organization’s goals and values and complied with relevant regulations.

When the organization switched to a new data management system, data migration projects were challenging. I managed data migration projects, ensuring that data was transferred accurately and efficiently from the old system to the new one.

Regular data audits helped ensure that the organization’s data management processes worked effectively and identified areas for improvement. I remember we regularly conducted audits of data collection, storage, and analysis processes to identify any gaps or inefficiencies. Data security was critical for organizations that handled sensitive information. We struggle to develop protocols for data security that protected the organization’s data from unauthorized access, use, or disclosure.

Finally, in order to ensure that staff members used data effectively and efficiently, I developed data training programs. These programs covered topics such as data collection, analysis, and visualization, as well as data management best practices.

2023-03-14

Data Management Data Science and Governance Digital Transformation Organizational change R&D

Data Management Risks and Rewards: A Comprehensive Guide to Maximizing Value and Mitigating Risk

by Massimo 2022-12-23

Data management is a critical aspect of modern business operations. The potential risks associated with low-quality data must be recognized, and proactive measures must be taken to ensure high-quality data that meets regulatory and ethical requirements. By prioritizing data quality, organizations can reap the benefits of improved decision-making, increased revenue, and decreased costs, while also safeguarding against potential risks.

As the role of information continues to grow in organizations across all sectors, it is essential to recognize that data not only represents value, but also risk. Inaccurate, incomplete, or out-of-date data can have serious consequences, leading to misunderstandings and misuse. Therefore, organizations must focus on ensuring the highest quality data, which is available, relevant, complete, accurate, consistent, timely, usable, meaningful, and understood.

However, despite efforts to maintain high-quality data, organizations often encounter information gaps – the difference between what is known and what is needed to make effective decisions. These gaps represent enterprise liabilities that can have profound impacts on operational effectiveness and profitability. According to recent statistics, organizations lose an average of $15 million annually due to poor data quality, and 84% of CEOs are concerned about the quality of data they use to make decisions.

Moreover, the increased role of information as an organizational asset has drawn the attention of regulators and legislators. From Sarbanes-Oxley, which focuses on controls over the accuracy and validity of financial transaction data, to Solvency II, which emphasizes the quality of data underpinning risk models in the insurance sector, data management has become a key area of regulatory scrutiny.

In addition, the last decade has seen a rapid growth in data privacy regulations, covering the processing of data about people across a wide range of industries and jurisdictions. Consumers are becoming more aware of how their data is used, and they expect not only more efficient operations but also protection of their information and respect for their privacy. Therefore, the scope of who our strategic stakeholders are as data management professionals can often be broader than might have traditionally been the case.

The balance sheet impact of information management arises when risks are not managed properly. Shareholders may vote with their share portfolios, regulators may impose fines or restrictions on operations, and customers may vote with their wallets.

The Leader’s Data Manifesto (2017) recognized that an “organization’s best opportunities for organic growth lie in data”. However, many organizations are far from being data-driven and mismanage their data and information technology. Only a few approach data strategically, leading to challenges in managing data. Let’s explore how committed leadership and involvement from everyone can unlock your organization’s organic growth potential.

I will present here the top 20 risks in data quality, in a clear and concise manner.

These risks can have a significant impact on a company’s operations and must be carefully considered mitigating their effects.

Incomplete Data: When important data points are missing or not collected, it can lead to inaccurate analyses and decisions.
Inaccurate Data: Data that is incorrect or out-of-date can result in incorrect conclusions or actions.
Inconsistent Data: Data that varies in format, structure, or definitions across different systems or departments can cause confusion and discrepancies.
Duplicate Data: Duplicate records can lead to erroneous data analysis and decision-making.
Insufficient Data Security: Insufficient security measures can lead to data breaches, resulting in significant financial and reputational damage.
Data Integration Challenges: Integrating data from different sources can be challenging, resulting in data inaccuracies and inconsistencies.
Poor Data Governance: Poor data governance can lead to a lack of clarity on data ownership, access rights, and standards, leading to confusion and data inaccuracies.
Lack of Data Standardization: Different data sources may use different terminologies or formats, leading to inconsistencies and inaccuracies.
Poor Data Quality Management: Inadequate data quality management can result in inaccurate, incomplete, and inconsistent data.
Inadequate Data Quality Monitoring: Inadequate monitoring can result in undetected data inaccuracies, leading to incorrect decisions.
Inadequate Data Quality Training: Inadequate training on data quality standards can lead to errors in data entry and management.
Data Bias: Bias in data collection or analysis can lead to incorrect conclusions and decisions.
Poor Data Entry: Poor data entry can result in incorrect data being entered into the system.
Data Silos: Data silos can lead to incomplete data analysis and decision-making.
Poor Data Cleansing: Inadequate data cleansing can lead to incomplete or incorrect data.
Insufficient Data Validation: Insufficient data validation can result in the use of inaccurate data for decision-making.
Inadequate Data Integration Testing: Inadequate testing can result in data integration issues and data inaccuracies.
Insufficient Data Quality Metrics: Insufficient metrics can result in an inability to measure data quality and identify improvement areas.
Poor Data Stewardship: Poor data stewardship can lead to data inaccuracies, breaches, and other data quality issues.
Insufficient Data Quality Tools: Insufficient data quality tools can lead to inadequate data quality management and decision-making.

Certainly, reducing the risks of data quality issues is critical to ensuring accurate and reliable data for decision-making. Here are 10 best practices that can help reduce the threat of the above items:

Develop a comprehensive data quality management plan: This plan should outline data governance policies, data quality standards, data quality metrics, and quality control procedures.
Standardize data collection and storage: By standardizing data collection and storage, organizations can ensure consistency in data format and structure, which reduces the risk of errors.
Conduct regular data quality assessments: Regular data quality assessments can help identify issues and take corrective action promptly.
Establish data quality metrics: Metrics should be established to measure data quality and identify areas for improvement.
Use automated data quality tools: Automated data quality tools can help identify and correct data quality issues quickly and efficiently.
Implement data validation rules: Data validation rules can help ensure data accuracy and completeness by checking data against predefined criteria.
Train employees on data quality: Employees should be trained on data quality standards and procedures to ensure accurate data entry and management.
Conduct data integration testing: Integration testing can help ensure data accuracy and consistency across different systems.
Establish data access controls: Data access controls can help ensure data security and prevent unauthorized access, reducing the risk of data breaches.
Monitor data quality regularly: Regular monitoring can help identify and correct data quality issues promptly, reducing the risk of inaccurate data being used for decision-making.

Reducing the risks of data quality issues is critical to ensuring accurate and reliable data for decision-making. Some best practices that can help reduce the threat of the above items:

Develop a comprehensive data quality management plan: This plan should outline data governance policies, data quality standards, data quality metrics, and quality control procedures.
Standardize data collection and storage: By standardizing data collection and storage, organizations can ensure consistency in data format and structure, which reduces the risk of errors.
Conduct regular data quality assessments: Regular data quality assessments can help identify issues and take corrective action promptly.
Establish data quality metrics: Metrics should be established to measure data quality and identify areas for improvement.
Use automated data quality tools: Automated data quality tools can help identify and correct data quality issues quickly and efficiently.
Implement data validation rules: Data validation rules can help ensure data accuracy and completeness by checking data against predefined criteria.
Train employees on data quality: Employees should be trained on data quality standards and procedures to ensure accurate data entry and management.
Conduct data integration testing: Integration testing can help ensure data accuracy and consistency across different systems.
Establish data access controls: Data access controls can help ensure data security and prevent unauthorized access, reducing the risk of data breaches.
Monitor data quality regularly: Regular monitoring can help identify and correct data quality issues promptly, reducing the risk of inaccurate data being used for decision-making.

2022-12-23

Data Management Data Science and Governance

Total Data Quality Management: A Comprehensive Approach to Data Quality

by Massimo 2021-08-15

Total Data Quality Management (TDQM) is a comprehensive approach to data quality that covers every aspect of data management, from data governance and data strategy to data modelling and data analysis.

TDQM focuses on ensuring that data is accurate, complete, consistent, timely, relevant, accessible, usable, transparent, and secure, and that it meets the needs of the business.

Total Data Quality Management is a comprehensive approach to managing data quality that ensures that data is accurate, complete, and consistent. It involves a range of activities, including data profiling and analysis, data cleansing, data governance, and data quality monitoring. TDQM helps organizations to improve their data quality, reduce data-related risks, and make better-informed business decisions. By implementing TDQM, organizations can ensure that their data is a strategic asset that supports their business objectives and enables them to compete in today’s data-driven business environment.

TDQM involves several key components, including data governance, data management, data strategy, data modeling, data analysis, data integration, data security, data privacy, data accuracy, data completeness, data consistency, data timeliness, data relevancy, data accessibility, data usability, data transparency, and data lineage. These components are interrelated and work together to ensure that data quality is managed effectively.

Data Governance

Data governance is the foundation of TDQM. It is the process of defining and implementing policies, procedures, and standards for data management. Data governance involves identifying the stakeholders, roles, responsibilities, and processes for managing data. It also involves defining the rules for data quality and ensuring that they are adhered to.

Data Management

Data management involves the processes and technologies for managing data throughout its lifecycle. This includes data integration, data modeling, data analysis, data quality assurance, data quality control, and data quality improvement. Data management ensures that data is accurate, complete, consistent, timely, relevant, accessible, usable, transparent, and secure.

Data Strategy

Data strategy is the process of defining the goals and objectives for data management. It involves identifying the business needs for data, the data requirements, and the data sources. Data strategy also involves defining the metrics for data quality and measuring the performance of data management.

Data Modeling

Data modeling is the process of defining the structure and relationships of data. It involves creating a data model that represents the data in a way that is understandable to the business. Data modeling ensures that data is consistent and that it meets the business requirements.

Data Analysis

Data analysis is the process of analyzing data to gain insights into the business. It involves using statistical methods and data visualization to identify patterns and trends in the data. Data analysis helps to identify data quality issues and to improve the quality of data.

Data Integration

Data integration is the process of combining data from different sources into a single view of the data. It involves identifying the data sources, mapping the data, and transforming the data to ensure that it is consistent. Data integration helps to ensure that data is accurate and complete.

Data Security and Privacy

Data security and privacy are critical components of TDQM. Data security involves protecting data from unauthorized access, use, disclosure, or destruction. Data privacy involves protecting personal information from unauthorized access or use. Data security and privacy are essential for maintaining the trust of customers and stakeholders.

Data Profiling and Analysis
Data profiling and analysis is an essential component of TDQM that involves the examination of data to understand its structure, content, and relationships. Data profiling is a process of gathering and analyzing metadata to understand the characteristics and quality of data. It provides a better understanding of the data to identify quality issues and helps to create a baseline for data quality improvement efforts. Data analysis, on the other hand, involves the use of statistical methods to detect patterns, trends, and anomalies in the data. The results of data analysis help to identify root causes of data quality problems, develop appropriate solutions, and monitor the effectiveness of the implemented solutions.

Data Cleansing
Data cleansing, also known as data scrubbing, is the process of detecting and correcting or removing corrupt or inaccurate data from a database. It is a critical step in TDQM that ensures that data is accurate, consistent, and complete. Data cleansing involves identifying and correcting errors such as misspellings, duplicates, and inconsistencies in the data. The process of data cleansing may involve manual or automated methods, depending on the nature and complexity of the data.

Data Governance
Data governance is the process of managing the availability, usability, integrity, and security of the data used in an organization. It involves the development of policies, procedures, and standards for data management, as well as the establishment of roles, responsibilities, and processes for data quality management. Data governance ensures that data is managed consistently across the organization, and that data quality is monitored and improved continuously.

Data Quality Monitoring
Data quality monitoring is the process of continuously monitoring and measuring the quality of data to ensure that it meets the required standards. It involves the use of metrics, indicators, and key performance indicators (KPIs) to measure the quality of data and identify areas for improvement. Data quality monitoring helps to ensure that data quality is maintained over time, and that data is fit for the intended purpose.

2021-08-15

Artificial Intelligence Data Science and Governance

How many fields in data science?

by Massimo 2021-06-27

Data Science is a research activity mostly

Data-driven scientific discovery is regarded as the fourth science paradigm

The twenty-first century has ushered in a new age that is coined as data science and big data analytics. Data-driven scientific discovery is regarded as the fourth science paradigm. Data science has been a core driver of the new-generation science, technologies and economy, and is driving new researches, innovation, profession, applications and education across both disciplines and business domains.

There are many scientific and technical challenges associated with big data, ranging from data capture, creation, storage, search, sharing, modeling, representation, analysis, learning, visualization, explanation, and decision making. Among the many data characteristics and complexities to be addressed, I mention the hybridization of heterogeneous, multisource, hierarchical,
interactive, dynamic, multidimensional, and quality-poor data mixed with real-time business operations, strategic planning, decision-making, value creation, and future developments.
The field of data sciences and big data analytics have been evolving from statistics since half century ago to broad areas including but not limited to data and signal analytics, knowledge discovery, information retrieval, machine learning, statistics, optimization, computing, and data management. The literature defined areas of Data Science that requires in depth knowledge and pure research to be effective. By synergizing the three big areas—statistics, informatics and computing, data science has been spreading to essential and specific areas such as

data intelligence and complexity analysis
representation, modeling, analytics, mining and learning including statistical and deep learning
computational intelligence including neural networks, evolutionary computing, fuzzy systems
neuroscience and linguistics
behavioral science and social and economic computing
uncertainty and optimization
system and modeling infrastructures and architectures
networking and interoperation
social issues including
privacy, security, trust, value and impact,
enterprises, services, applications, solutions and systems
simulation, visualization and explanation

2021-06-27