A tool to dynamically merge corresponding weather data to crop data
Xinwen Yu, Seishi Ninomiya, Atsushi Yamakawa
National Agriculture Research Center, Tsukuba, Japan
Dead Storage Data Issue A lot of digital data sets are continuously produced in
agricultural experimental stations Using ordinal software such as spread sheet applications But they are likely to be kept in local stations and scientist level
The data sets are isolated and hardly integrated among different locations
How to ease data publication for merging and sharing for end users
Different resource integration issue Separated crop data are hard to be
integrated with different resources, e.g. weather data
Heterogeneity e.g. Models constructed using local
data are only applicable locally How to integrate crop data with
weather data under user’s request
If merging and sharing are possible,
End users can freely combine separated data sets from different locations and perform analysis on them
Datamining over the huge amount of data sets becomes real and we can possibly find out unknown facts
Integration with completely different resources e.g. weather data becomes also possible
Constructing global model also becomes possible
With Web-based data sharing & integration
Once you upload experimental data in spreadsheet file (e.g. MS Excel), data automatically become sharable over the Internet among different locations
Crop data can be integrated with weather data No skill is required Just a part of everyday data management Uniformity of tables is not required Low cost
Internet
DBMS
Application Server
Servlet Container
EJB Container
Application
Concept
Multi-location data sharing and integration Integration with weather data
MetBroker
Basic structure of application
Google Map
Client
Brow
ser
Controller
Crop DBCrop Data Service
Model
ServicesWeb application
MetBroker
AMeDAS
AMeDAS
AMeDAS
View
System components
Java Runtime Environment 1.5.04
PostgreSQL7.4
JRE supported OS
JBoss-4.0.3
EJB3.0(DBMS abstraction)
Struts1.2(Web Interface)
・ IE, Firefox, etc.・ Excel2002, newer
Main menu Very simple menu structure Simple and clean user interface
Data upload
Crop Data ServiceEJB3
SourceXML
CropData
HistoryData transforming
Data validating
XSLT style sheet
Data Schema
a
b
c
Crop Database
a. Transformation of source data file using style XSLT style sheet, storing source data file in XML to SourceXML table.
b. Validating transformed data XML using data schema c. Data service binds data XML to data object (JAXB), maps data object to data records t
o CropData table; stores uploaded information including user name, spreadsheet file name, uploading time, count of data record to History table.
Upload history
Web Application
Crop Data Service EJB3
Crop db
Specifying query conditions then executing data query.
Browsing and/or download queried crop data
Data Query
AMeDAS
Web Application
Crop Data
Location Table
Crop Data Service EJB3
MetBroker
ab
AMeDASAMeDAS
cd
Data integration
Location registration
Weather station selection
Discussion
37 spreadsheet files uploaded 151072 records 6486 varieties 184 properties 256 different experiment locations 1980-2004
Available to open society
Data integration is realized in our brokerage based Data Grid MetBroker is the core of the Data Grid
The application demonstrates the power of Data Grid
This approach can be used to all other crop data sharing and integration
Crop modelers are using this application to develop robust crop models
Thank you for your attention