About the project
Agriculture is the top of priority sectors in Ethiopia. It employs more than 80 percent of the population and the source for foreign currency. The government of Ethiopia set out a development plan to enhance agricultural production and productivity as one of the major strategic pillars. Unless the country modernizes its agricultural sector by utilizing its knowledge resources, it would be challenging to feed unlimited people for unlimited years with limited resources.
There are different technologies to easily capture agricultural data. These technologies include IOT, Sensors, RFID to track livestock and food, drones, satellite sources, mobile devices, legacy databases by different agricultural departments and more. There is also a significant amount of data in the form of annual reports, project reports, policies and regulations using word processing software. The agriculture sector has already a huge data. NASA, for example, archives more than 7000 types of HDF, netCDF, GeoTIFF, FAST and ASCII data. Data gathering is an easy part. The hard part is bringing it together all these data and analyze it to generate hidden patters and knowledge that will be used for decision making activities and problem solving.
This project is initiated to develop and operationalize an agricultural data ecosystem in Ethiopia (aka. ag-data hub) that integrates disparate systems that are available within Ministry of Agriculture and other partner organizations which are at the federal and regional levels. The main functionality of the system is to extract data from different sources and then transforms the data and then stores in the Agri-data hub. The system has different modules to support these basic functionalities.
Approach
The development of the Ag-Datahub has gone through different steps
Requirement Design
Co-design of requirements and system architecture with project stakeholdersDeveloping and refining the system requirement specification document
a well•documented list of requirements, a high-level requirement specification document that shows the rationale, scope, the methodology to be followed, the nature of the product is developed following the IEEE requirement specification guideline.Design solution architecture
Designing the technical solution architectureMetadata Design
Designing metadata standards and data architecturesImplementing harvesting pipelines
Implementing data harvesting pipelines and ServiceDesign data access points and interfaces
Data access points and interfaces are being designed to make the national ag-dotahub accessible for both humans and machines alike
Datahub Services
Gather data from disparate sources, Integrate data into a unified view for data consumers, Prepare data for analytics and reporting, Manage data pipelines for a continuous flow of data from sources to the Data HUB, Manage the complete infrastructure for the collection, processing, and storage of data.
key tasks: Connect to data sources. Transfer data from these data sources to the data platform in streaming, batch, or both modes. Maintain information about the data collected in the repository.
Store data for processing and long-term use. Transform and merge extracted data, either. Make data available for processing in both streaming and batch modes
Validating, transforming, and applying business rules to data, Read data in batch or streaming modes from storage and apply transformations. Support popular querying tools and programming languages. Scale to meet the processing demands of a growing dataset. Provide a way for analysts and data scientists to work with data in the data platform
Delivers data to data consumers who consume this data through interactive visual representations, such as dashboards and analytical reports. Data users further process this data for specific use cases. Other applications and services that may need this data as input for further use.
About the Initiatives
The AICCRA Ethiopia project through its implementing center, the Alliance of Bioversity and the International Center for Tropical Agriculture (CIAT), went through different engagements with national partners, particularly with the Ministry of Agriculture, to map key actors/data holders, assess the nature of data and meta-data of each institute, and analyze the accessibility of those data. The Accelerating Impacts of CGIAR Climate Research in Africa (AICCRA) program is supporting the development of an integrated ag-data hub in Ethiopia.
The program is considering to co-develop a dedicated, and publicly owned and operated - one-stop shopping national ag-data hub. Further, to make sure that the development of the Ag datahub system aligns with the government plan, a high-level discussion was held with key personnel in the Ministry. ACATECH Technology was assigned to lead the consortium and development of the data hub for Ethiopia. The company works with different partners in the country including those involved in system design and subject matter specialists.
Features
- Security
Access to the system is properly managed to control any unauthorized attempt to access the system. Access by authorized users also be controlled to make sure users are presented with data that they have a privilege to.
- Scalability
The datahub is designed in such a way that it accommodates the growth of data without compromising performance
