software development in agricultural computer science ... · the cathedral & the bazaar (eric...

Software Development inAgricultural Computer Science:

Status and PerspectivesEildert Groeneveld

[email protected]

Rodica, April 15th. 2004

Institute for Animal Breeding, FAL, Mariensee

Software Development in Agricultural Computer Science: Status and Perspectives – p. 1

Outline

1. Objectives in Computer Science in Agriculture

2. The Framework

3. Thesis

4. the “so what?” projects

5. examples of dead-end developments

6. possible solutions and development paths

7. examples of the OS development model

8. Open Source development

9. outlook


Objectives in Agr. Computer Science

Facilitate IT solutions in Agriculture

quick development and implementation of new solutions

requires research and development

essentiell framework:

cumulative mode of developmentleads to accelerated developmentis based on original research

progress resulting from the general development inSW/HW does not count


The Framework

Information processing in Agriculture:

in the scientific area:

design and evaluation of experimentsdevelopment of computing algorithmswell defined inputs -> problem solution -> Output(IPO)


The Framework ff


on farm area:

solutions to local problems: optimization of feedrations (IPO)area of global information: collection, managementand evaluationmerging data from various sources:

lots of land, soil samplesperformance recording: milk, growth, reproduction

across farms:

aggregation of data from many sources (InfoSys)in animal production: BLUP genetic evaluation


The Framework fff


animal breeding, crop production, agriculturalengineering, agricultural economics

different kind of systems (IPO ⇔InfoSys):

IPO: production of data -> problemsolution->decision->throw data away

InfoSys: integrated data utilization: breeding, milkrecording, labs, AI, utilization of the WEB,networking of farms


IPO vs InfoSys

conceptual differences:

IPO:

simple well defined inputscomplex processing but usually in a simple programenvironment (F90, C, ...)well defined outputsprocessing layer is disconnected from the inputs


Examples IPO

statistical procedures through use of packages (SAS,R...)

analysis of inbreeding (SW)

computation of BLUP (SW)

estimation of covariance components (SW)

etc.


IPO vs InfoSys ff

conceptual differences:

InfoSys:

many sources of data from different origin/systemsas a result: different SWas a result: different operating systemsincludes data transferdifferent data structures among different problems(organizations)everything is connected with everything


Examples InfoSys

gene banks in plants and animals

e-Commerce Systems

query and information systems

integrierted data bases: plants/animals

many components:

Webserver,DBMS,programming languages,OS


Thesis

IPO:

1. many developments with large impact: DFREML,MTDFREML, ASREML, VCE, ABTK, PEST

InfoSys:

1. many developments that solve ONE problem

2. are thus irrelevant for the industry as a whole:

3. the “So What?” projects.


The Problem: An Example

WCGALP Australia in Section :”Computing Techniques in Animal Breeding”

total of 26 contributions

17 IPO

4 InfoSys:

NZ sheep improvement schemeQTL detectionWEB Animal Recording SchemeAMNOS Dairy Sheep Managementall SW


Analysis of the Problem: own examples

1970: central database for the BHZP:

HP-1000network type databaseelectronic on-farm data recordingelectronic automated data transferremote program execution

Effect:

solves the BHZP problem, but no other despitepublicationsorry to say: SW project


Analysis of the Problem: own examples ff

End 1980:

new attempt at generalization:ORACLESUN-OSpig breding program Sloveniatransfered to Chinese Swine Herd at U of I (USA):no transfer beyondsolved a problem in Slovenia and U of I but nothingelse despite Master’s thesissorry to say: SW project


A Key Experience

meeting of developers of integrated databases

different countriesdifferent external circumstancesremarkable:

different systems and tools (dBase, Clipper,Oracle...)totally different development pathsevery system a completely new developmentwithout reference to previous work (literature?)no way to use software acrossnothing worked really well


Critical Question

How manydevelopment projectsin the different areas

are of the kind:“So What”?


What to do?

the basic problem:

solution of ONE problemthus: not transferablebecause solution of one problems includes all sideconditions:

operating systemRDBMSsystem tools and software

this is hardly ever an issue in ONE problembut strong restriction on transfersoftware is generally not available


What to do? ff

solution can only lie in:

generalization: find the Poodle’s corealso: never develop for ONE case, but always formanyparallel implementationimplement a layered system which allows replacingcomponents (non monolithic)release software


A New Development Path

started as “Integrated Database of PerformanceRecords in Pig Populations”

any animal identification systemany data structureinstallation free of licencing costsscales from on-farm to nationaldevelopment of procedures for development andmigrationdevelopment of software toolboxdevelopment of general evaluation software


A New Development Path ff

avoid proprietary software

use standards (SQL-99)

use Open source

developer group

implementation in at least two environments


Development

Strategy/Procedure

can be used independently from software⇒migration steps of an information system

Software

for the implementation of the procedures

generic solution for BR


Closed or Open Development?

IPO can work without program modification

InfoSys: different data structures require softwaremodification⇒ System without source code not conceivableclosed development:

few people have the knowledgerestricted pool for future development

need to create development capacity⇒OpenSource


Open Source Development

The Cathedral & the Bazaar(Eric S. Raymond)

“given enough eyeballs, all bugs are shallow”

big project:

EMACS, LINUX, FETCHMAILKDE, GNOME

requires infrastructure:

Internet connectionSoftware management system (CVS)


Open Source: Advantages

get help with debugging

expand developers base

enhances future system safety through expandedknowledge base

help during conceptual development

cost effective developer and test capacity via theInternet

the world is the limit (cooperation space)


Implementers

Institute of Animal Production, Irene, South Africa

Landesanstalt für Tierzucht, Köllitsch

University of Ljubljana, Slovenia

Research Institute Nitra, Slovakia

University Debrecen, Hungary

Veterinary Academy Kaunas, Lithuania

University Stara Zagora, Bulgaria

FAO

University Göttingen

TZ, Mariensee


Current Projects

PISSA South Africa, Pigs, Cheetah

Beef South Africa, Cattle

PIS Lithuania, Pigs

CryoDB Netherlands, Germany, Genebank

MINIPIGS Göttingen, pigs

MOLABIS Germany, DNA Material

SCHAF-OT Germany, Sheep

Saxonian Pig System, Germany

EFABIS (FAO, F, PL, D), Biodiversity Database

a number in Slovenia: Rabbits, Horses, Cattle, Pigs, ..


Platforms create Utility

many developers in CC, Research Inst. but SWgenerally not usable in InfoSys

on generic platforms software can be used:

Population reportZWISSS...

cumulative development model


Population Report 1

��

��

��

� � � � �� ! � � "#� ��$ % & � $ ' � �( � $ � � �

") *+ , %- ./ � � � � � 0 01 /

2354 6 %� % � / 0 03

78 9 :; < < =

>@? A ? B ?C 9


Population Report 1 ff

� ��

� "! #! $ % &

' (*) + , ! $ - .*/ 0 $! / % # 1 1 % +/ 0 # $! 2 $ - 1) 3 4 0 - # ,*5 5 ! % $

� 67 � � �8 9 � � : 7 � � � � 6� � � ; �� 9 < � 9 �� 9= � � �� > 7 � � �8 � 9 > � ? � 7 9 � � @�

6 �A � 9 � � : 7 � � 8 � �B � �� C �� :� � D � � 6� E � � �8 �� 9 < � 6� � � � :� : 7 � � � � 6� � 7 ��

< 9 : : 9= � > �B � 9= � � � A � �B � �F � 7 9 � 6 �� 6 �� F 9 : � ; � � � � 6� B �� 8 �� C � 9 ��

� � > � 9= � DHG

� � � � � > 7 � �� A 7F � G 6� �� = � F 9 � � � � 6� � � ; �� 9 < � 9= � � � > � 9 �� = 6 98 �� 7F 78 � �� 7 � �� A 7F � � > � � 7 �I � 6� F 9 � � � �8 9 � > 7 � I B � ��

� � 7 � :7 � �� G � 6 7 � 7 � � 6� � 9 � � : � � ; �� 9 < � 9= � �� > � 9 �� = 7 � 6 : 7 � �� 7 � � 6�

I 7 A � � B � ��

�� I 7 A � � � 6� � � ; �� 9 < � 9 �� > � 9= � 6 �A 7 � I �� :� F �� > 9 J�8 � 7 �I 7 � � 6�

I 7 A � � B � �� :�F � 7 9 � 7 � >� E � � > �� 6 �A 7 �I � E � � � �� A 7F � � � 6 � �*K � 67 � 7 �

� 6� � � ; �� 9 < � 9= � �� > � 9 �� = 7 � 6 � E � � � �� A 7F � 7 � � 6� I 7 A � � B � ��

� � �� : :� �G � ; �� 9 < � 7 �� > > � ;� 7 � ��8 � 9 > �F � 7 9 � �B B � �� C � @ D

� � �� ; �� 9 < � 7 � � � � � ; �� 9 < > � ; �

�� A 7F � � : 7 � �� 9 J�8 � � �� :� � � � A 7F � � : 7 � �� 9 J�8 � � � � :�� L M L � � � � � �

� LN � � � � �

� LN � O � � O � �

� LN O � O � � O �

� LNP O Q � Q M �

� LN Q N O P � � M Q

� LN M O M O � � � � P

� LN N � � P Q � O � � � �

� LN L � � � � N � M � O � O

� L L � � O � � � � � L � Q � O

� L L � � � � L � � ON O Q � Q

� L L � � Q � L � N P N ON � O

� L L � � � � � � Q M � Q O � N

� L L O � � � � � � MN M Q �P

� L LP � N � N �P M M N � � �

� L L Q � Q �P � � P � P � � L

� L L M � � � O L � � O � � �

� L LN � M � � N


Population Report 2

��

��

��

� ! � � �" �� # � � $ ��% & � ')( �� (� & �� $ ��% & �* " ! + & � � �,

-./ -/ 0� " ! � &� � 1 & �� !� �

243 5�" ��6 7 7 -

89 :9 ;9 <= > <?@ @ ACBD EB

F


Population Report 2 ff

� � ��

� ! � � �� "# � $ ! �% & '( �% � ( � ) ) $( �� % �* % & )� +� � &� !, , � % - � & .

� � % � � $ ! �% & ' ( �% � ( � � $ ! �% & ' ) $(

( �% / � +� ( � �� % ( & 0( * % � ( � � � ( �% / � +� ( � �� % ( & 0( * % � ( � � �"1 2 3 41 � 2 5 2 " 2 " 1 4 " � 3

"1 2 5 2� 4 � 5 2 " 2 3 "� � 1 "

"1 2 4 � " 51 4 4 �� 5 " � � 2 4

"1 2 � � " 51 5 � " 4 " "� " 41

"1 2 2 4� 4� 5 4 " 5� "� � 2�

"1 21 4 � 5� 5� " �1 " � 4 � �

"1 1 � 31 � 1 5� " � � � " 4�

"1 1 " 4 � 5 2 5 " " � � 2 4 4 2

"1 1 � 4 � 4� 5 3 " 3 5 2 3 4 4

"1 1 � 3 � 3 4 31 2 4 41 5 5

"1 1 3 " " " " � " 5 " 4 �

"1 1 5 3 " � � 4� 41 � � 2 �

"1 1 4 2 4 5 � � 3 5

"1 1 � " 5 " � " 2 3� � � ��

"1 1 2 �1 � � � 2 4� � � � 3

"1 1 1 �� " 3 � � � 2 � "

�� 3 " 3 � 5 � 4 �� 2

�� " �1 �� 5 � 3 ��

�� " � �� " � �� " "�

�


Publications

is SW development in InfoSys or IPO research?

I think it is. Need to change attitudes.

general problem with OS projects

may be: we are part of the problem?


Summing Up

Commercial Development: solve ONE problem

Research in CS: develop generic solutions

research in this area is not uncontested

clean definition of layers

Open Source

avoid proprietary solutions

push for a cumulative development model


software development in agricultural computer science ... · the cathedral & the bazaar (eric...

Documents