linear measurement accuracy of dji drone platforms and ... · determine the optimum flight settings...
Post on 05-Jun-2018
214 Views
Preview:
TRANSCRIPT
-2-
DroneDeploy|1045BryantStreet,Suite300|SanFrancisco,CA94103|dronedeploy.com
TableofContents
Abstract......................................................................................................................3
Introduction................................................................................................................4GoalsofThisStudy..............................................................................................................4ExploringBestPracticesforPhotogrammetricDataCollection............................................4DroneModelsUsedinThisStudy........................................................................................4CameraSensors...................................................................................................................5
Background.................................................................................................................6StandardDeviationofLinearMeasurementsfromPhotogrammetry...................................6GuidelinesforProducingAccurateAerialMapswithDrones...............................................6
Methodology...............................................................................................................6EstablishingaGroundControlSystem.................................................................................6CollectingAerialData..........................................................................................................7CapturingGroundControlPointData..................................................................................7
KeyFindings................................................................................................................9OnAverage,LinearMeasurementsareAccuratetoWithin1.1%.........................................9HigherResolutionCamerasReducedAverageMeasurementErrorby0.33%.......................9FlyingLowImprovedMeasurementAccuracyby0.35%......................................................10Together,CameraResolutionandAltitudeHalvedAverageMeasurementError................11VisualizingMapResolutionbyAltitude..............................................................................12GroundControlPointsImprovedMappingAccuracy10x....................................................12
Conclusion.................................................................................................................13UseHighResolutionCamerasforIncreasedMeasurementAccuracy..................................13FlyMissionsatLowAltitudeforHigherMeasurementAccuracy.........................................13GroundControlPointsProducetheMostConsistent,AccurateResults..............................14AltitudeMattersWhenMappingwithGroundControlPoints............................................14
AdditionalDroneDeployResources...........................................................................15AGuidetoUsingGroundControlPointswithDroneMappingSoftware...................................15DecidingIfYourDroneMappingProjectNeedsGCPs..............................................................15CapturingGroundControlPoints(GCPs).............................................................................15GCPRequestChecklist........................................................................................................15DroneDeployIn-BrowserGCPTaggingWorkflow................................................................15
Appendix...................................................................................................................16
-3-
DroneDeploy|1045BryantStreet,Suite300|SanFrancisco,CA94103|dronedeploy.com
AbstractDronesarequicklybecomingthego-tomeansforthecollectionofon-demandaerialimageryacrossindustriessuchasconstruction,surveying,insurance,andmining.Photogrammetryallowsustodigitalizethephysicalworldandusethatdatatosolvesomeoftoday’stoughestchallenges.Intheprocess,iteliminatestheneedtomanuallycapturedataindangerousareassuchasindustrialjobsites,quarries,roofs,andotherelevatedstructures.Photogrammetryreliesoncamerastomeasurereal-worldobjectsandturn3Dspaceinto2Dmaps.Thisrequiresphotogrammetrysoftwaretoidentifythelocation,orientation,andmovementofthecameratocalculatethepositionofthree-dimensionalpoints.Withoutaphysicalrelationshipbetweenthecameraandthesubjectbeingmapped,itisdifficulttoquantifytheaccuracyofphotogrammetricoutputs.Therefore,toproperlyquantifyerrortheoutputsmustbeground-truthedagainstknownvalues.Camerasarephysicaldevicesthatintroduceerrorsintothedatacaptureprocess.Acamera’s2Dimageisn’tatruerepresentationofthephysicalworld.Thisisbecausecamerabodiesandlensescannotbemanufacturedperfectly,whichcreateerrors—suchasdistortedlinesinaphoto—thatphotogrammetrysoftwaremustcompensatefor.Theseerrors,evenwhencompensatedfor,cancreateinaccuraciesinlinearmeasurementsmadeonaprocessedmap.InthisstudyDroneDeployhasinvestigatedwaystoimprovemappingaccuracyandputtogetherasetofbestpracticestobeusedwhenmakinglinearmeasurements.Totesttheaccuracyofthemeasurementsmadeusingmapsgeneratedfromdatacapturedwithindustry-standardDJIdroneplatforms,DroneDeployestablishedagroundcontrolsystemontheroofofitsoffice.Then,morethan80flightswerelogged—eachexploringdifferentflightaltitudes,cameras,andphotooverlapsettings.TheimagescollectedwerethenprocessedinthecloudusingtheDroneDeploymapengine.Thedatasetswereanalyzedandusedtocalculatetheaveragemarginoferrorformeasurementsofknowncontrollengths.Usingtheresults,DroneDeploydeterminedthatusingaperformancecameraandflyingatlowaltitudewithhighimageoverlapproducedmapswiththebestlinearmeasurementaccuracy.
-4-
DroneDeploy|1045BryantStreet,Suite300|SanFrancisco,CA94103|dronedeploy.com
IntroductionGoalsofThisStudyWhenitcomestodronedata,thereisonemetricthatmattersmost:accuracy.Overthelastfewyears,asdronetechnologyhasadvanced,sotoohastheabilityfordronemapstobehighlyaccurate.Butphotogrammetricaccuracyispoorlyunderstood—particularlyamongthosewhoarenewtoaerialmappingorjustenteringthecommercialdroneindustry.Tohelpshedlightonthistopicandprovidebusinesseswithaccesstoreliabledata,DroneDeployconductedanexperimenttoassesstheoveralllinearmeasurementaccuracyofdronemapscapturedbystandardDJIdroneplatformsandprocessedinthecloudusingtheDroneDeploymapengine.Thegoalofthisstudywastoprovidequantitativedataandguidelinesforwhatuserscanexpectfromlinearmeasurementsonmodelscreatedwithcloud-basedphotogrammetrysoftware.Thisstudyalsoincludedtherelativeaccuracyofdatacollectedandprocessedwithandwithouttheuseofgroundcontrolpoints(GCPs).TheresultsprovideinsightintotheaccuracyonecanexpectfromeachmethodanddatatoaidindecidingwhetheraDroneDeploymappingmissionrequirestheuseofgroundcontrolpoints.ExploringBestPracticesforPhotogrammetricDataCollectionThispaperprovidesguidanceonhowtoimprovethequalityofinputdatacollectedusingDJIdronemodelsandcameraswhenproducingmapsontheDroneDeployplatform.Thisstudy,comparesimageresultsfromawiderangeofhardwareoptionstodeterminetheoptimumflightsettingsforproducingmapsthatcandeliveraccurate2Dmeasurements.Theresultscanserveasguidelineswhenselectingflightparametersformappingmissionsincluding:altitude,sidelap,andfrontlap.DroneModelsUsedinThisStudyTheDJIdronemodelschosenforthisexerciseincludedtheMavicPro,Phantom3Pro(P3P),Phantom4Pro(P4P),Inspire1(withDJIX3Camera),Inspire1Pro(withDJIX5Camera),andInspire2(withDJIX4sCamera).TheotherDJIPhantom3modelsandtheDJIPhantom4wereexcludedbecausethesemodelssharethesame12MPcamerasensorincludedinthePhantom3Pro.DroneDeploychosetoconductthisstudywithDJIdronesbecausetheyarethenumberonecameraandhardwareplatformusedbyprofessionalsinthecommercialdroneindustry.AsthesedronesaretheprimarychoiceofDroneDeploycustomersacrossindustriesincludingconstruction,agriculture,inspection,surveying,andmining,thisdatasetprovidesDroneDeploy’slargestcustomerbaseinsightintotheaccuracythatiscapablewiththeirhardwaresolutionofchoice.
-5-
DroneDeploy|1045BryantStreet,Suite300|SanFrancisco,CA94103|dronedeploy.com
CameraSensorsTherewereavarietyofsensorsusedtocollectdatainthisstudyrangingfrom12MP,1/2.3”to20MP,1”.ThisselectionrepresentsthefullrangeofDJIcamerasavailableonthemarkettodayandincludesbothmechanicalandelectroniccamerashutters.ThesensorspecificationsandcharacteristicscanbefoundinFigure1andTable1.
Figure1.CameraSensorSizes
Table1.CameraSensorSpecifications
Drone Megapixels SensorSize FieldofView ShutterType
P3P&I1(X3) 12MP 1/2.3” 94° Rolling
MavicPro 12MP 1/2.3” 78° Rolling
P4P&I2(X4s) 20MP 1” 84° Mechanical
I1Pro(X5) 16MP 4/3” 72° Rolling
-6-
DroneDeploy|1045BryantStreet,Suite300|SanFrancisco,CA94103|dronedeploy.com
BackgroundStandardDeviationofLinearMeasurementsfromPhotogrammetryPubliclyavailabledatarelatedtothestandarddeviationoflinearmeasurementsfromphotogrammetrymodelsislimited.Existingstudiestypicallylookattheaccuracyofstockpilevolumes,anddoesnotincludedataspecifictolinear,point-to-pointmeasurements.Thismakesitincrediblydifficulttounderstandthemarginoferroronecanexpectfromdistanceandareameasurements.Inaddition,canonicalresearchonphotogrammetricaccuracyfocusesontheuseofGCPcheckpoints.ThismeanstheaccuracyismeasuredasadifferencebetweenthegeographicpositionofacheckpointasrecordedbyprecisionGPS,andthetheoreticalpositionofthatcheckpointascalculatedbythephotogrammetrysoftware.Thismethodhasbeenfavoredbecauseitisthebestwaytoisolateexperimentalvariables,andconsideredtobethemostscientificallyrigorous.However,becauseacheckpoint’saccuracycanonlyapplytoasinglepointinspace,thedatadoesapoorjobofevaluatingtheaccuracyoflinearmeasurements.GuidelinesforProducingAccurateAerialMapswithDronesTodate,therearefewresourcesthatprovideguidelinesregardingthecaptureandprocessingofaerialdataforcommercialuse.Thosethatdoexistdonotprovideharddatatobackupsuggestions,whichleavesmappingprofessionalswithlittleevidencetoguidedatacapturemethods.
MethodologyEstablishingaGroundControlSystemTotesttheaccuracyoftwo-dimensionalmeasurements,DroneDeployestablishedagroundcontrolsystemontheroofofits1045BryantStreetoffices.ThisisshowninFigure2.Inadditiontogroundcontrolpoints,thiscontrolsystemdefinedcontroldistanceswithknownlengths.Thesecontroldistancescouldthenbecomparedwiththeoreticalmeasurementsmadewithphotogrammetrysoftware.
-7-
DroneDeploy|1045BryantStreet,Suite300|SanFrancisco,CA94103|dronedeploy.com
Figure2.GroundControlSystem
Thegroundcontrolsystemusedthree1’x1’checkeredmarkersthatwereplacedinalargerightangled“L”shapealongthebuilding'sprimaryaxis,withafourthmarkerwasaddedasalengththatbisectsthe“L”axis.ThecontroldistancesbetweeneachmarkerweremeasuredmanuallyusingametaltapemeasureandcheckedwithaLeicaLaserDistanceMeter.CollectingAerialDataAerialdatawasthencollectedovermorethan80individualflightsusingDJIdronesandtheDroneDeploymobileapplication.Atotalof4flightswereflownat66ft.,100ft.,200ft.,and400ft.abovethesurfaceoftheroofusingeachDJIdronemodel,andthenidenticallyprocessedwiththeDroneDeploymapengine.Slightvariationsinlighting,flightpath,andaltitudewereincludedtosimulaterealworldvariationincollectedmappingdata.CapturingGroundControlPointDataTocollectthegroundcontrolpointdata,5additionaltargetsmeasuring3”x3”wereplacedontheroof.ThegeographiclocationsofthecenteroftheselargertargetswerethenrecordedwithanEmlidReachRSandTrimbleCatalyst.Theseglobalnavigationsatellitesystem(GNSS)receiverscanbeseeninFigures3and4.AprocessedaerialmapofthegroundcontrolsystemsetupcanbeseeninFigure5.Oncethegroundstationwassetup,aDJIPhantom4Prowasusedtomaptheroofoncemoreatelevationsof66ft.,100ft.,200ft.,and400ft.
-8-
DroneDeploy|1045BryantStreet,Suite300|SanFrancisco,CA94103|dronedeploy.com
Tobetterunderstandthestandarddeviationandaveragemarginoferror,astatisticalsamplewastakenusingthePhantom4Pro.FlyingidenticalmissionplanswithGCPsatanelevationof66ft.,atotalof6flightswerecompleted.Similarly,afinaldatasetwascreatedwithoutGCPsbyflyingthePhantom4Proat66ft.anadditional10times.Thismethodwasadoptedtoapproximatethevariationpresentinlinearmeasurementaccuracywhenmappingonelocationunderthesameconditions.
Figure3.EmilReachRS Figure4.TrimbleCatalyst
Figure5.AerialViewofGroundControlSystem
-9-
DroneDeploy|1045BryantStreet,Suite300|SanFrancisco,CA94103|dronedeploy.com
KeyFindingsOnAverage,LinearMeasurementsareAccuratetoWithin1.1%WhencomparingthedatacollectedacrossalltheDJIdronemodelsincludedinthestudy,theaveragemarginoferrorforallflightswasfoundtobe1.1%.Bythiswemeanthevalueoftheaveragemeasurementerrorwas1.1%ofthevalueofthecontrollength.Thus,whenmeasuringa100ftlengththeaverageerrorwouldbe1.1ft.TheseresultscanbefoundinTable2.
FlightAltitude(ft.) Measurement1(40ft)Error(ft.)
Measurement3(50ft)Error(ft.)
Measurement2(100ft)Error(ft.)
66 0.25 0.27 0.54
100 0.35 0.39 0.85
200 0.56 0.79 1.34
400 0.67 0.88 1.44Avg.MeasurementError(ft.) 0.45 0.57 1.04
Avg.MeasurementError(%) 1.12% 1.14% 1.04%
Table2.MarginofErrorWithoutGCPs
HigherResolutionCamerasReducedAverageMeasurementErrorby0.33%Thedatashowsaclearcorrelationbetweenhigherresolutionimageryandhighlyaccuratemaps.Forexample,themapsproducedwiththe12MPP3Pcamerayieldedanaveragemarginoferrorof0.72ft.acrossmeasurementstaken,whereasthemapsproducedwiththe20MPP4Pcamerayieldedanaveragemarginoferrorof0.46ft.Bycomparingtheaveragemarginsoferrorforeachcontrollengthasapercentage,itwasfoundthattheaveragemeasurementerrorfor20MPcamerawas0.33%lessthanthe12MPcamera.ThisdatacanbeseeninTable3.While0.33%mynotseemsignificant,rememberthatwouldbea0.33ft.(~4in.)reductionoferrorfora100ft.measurement.Theaveragemeasurementerrorsbetweenthe12MPmapsand20MPmapsshowedthaterrorwouldincreaseproportionallytocameraresolutionwithacorrelationcoefficientof0.33.Between12MPand16MPmaps,thiscorrelationcoefficientwascalculatedat0.56.
-10-
DroneDeploy|1045BryantStreet,Suite300|SanFrancisco,CA94103|dronedeploy.com
Measurement 1
(40ft) Error (ft.) Measurement 2
(100ft) Error (ft.) Measurement 3 (50ft) Error (ft.)
Flight Altitude (ft.) P3P (12MP)
P4P (20MP)
P3P (12MP)
P4P (20MP)
P3P (12MP)
P4P (20MP)
66 0.34 0.19 0.73 0.46 0.34 0.245
100 0.39 0.30 0.88 0.84 0.41 0.31
150 0.47 0.60 1.09 1.52 0.38 0.64
200 0.32 0.89 0.64 2.18 0.52 1.04
300 1.55 0.12 3.58 0.36 1.94 0.06
400 0.92 0.33 1.99 0.68 1.13 0.41
Avg.MeasurementError(ft.) 0.49 0.40 1.06 0.49 0.60 0.49
Avg.MeasurementError(%) 1.22% 1.00% 1.06% 0.49% 1.20% 0.98%
Table3.CorrelationBetweenResolutionandMeasurementAccuracy
Theaveragemeasurementerrorsbetweenthe12MPmapsand20MPmapsshowedthaterrorwouldincreaseproportionallytocameraresolutionwithacorrelationcoefficientof0.33.Between12MPand16MPmaps,thiscorrelationcoefficientwascalculatedat0.56.FlyingLowImprovedMeasurementAccuracyby0.35%Thedatasuggeststhatflightplansconductedwithloweraltitudeswillproducemapswithhigheraccuracythanthoseflyingathigherelevations.Theaveragemeasurementerrorsshowedthaterrorwouldincreaseproportionallytoflightaltitudewithanaveragecorrelationcoefficientof0.42.ThiscorrelationisvisualizedbelowinChart1andTable4.Therewasa0.35%improvementinmeasurementaccuracyformapsflownat66ft.elevationcomparedwiththoseflownat100,200,or400feetofelevation.HeightCorrelationFactor
Measurement1(40ft.)
Measurement2(100ft.)
Measurement3(50ft.) Average
Phantom4Pro 0.466 0.453 0.419 0.45
Inspire1Pro(X5) 0.355 0.334 0.564 0.42
Inspire1(X3) 0.548 0.512 0.807 0.62
Phantom3Pro 0.355 0.295 0.378 0.34
Total 0.46
Table4.FlightAltitudetoMeasurementErrorCorrelationCoefficients
-11-
DroneDeploy|1045BryantStreet,Suite300|SanFrancisco,CA94103|dronedeploy.com
Chart1.Phantom4ProMeasurementErrorvs.FlightAltitudeTogether,CameraResolutionandAltitudeHalvedAverageMeasurementErrorWhenflyingatloweraltitudesandwithahigh-resolutioncamera–theaveragemarginoferrorwasreducedto0.64%.Thisisnearlyhalftheaveragemarginoferrorforalldronesatallaltitudes.TheseresultscanbefoundinTable5.
P4PFlightNumber Measurement1(40ft)Error(ft.)
Measurement3(50ft)Error(ft.)
Measurement2(100ft)Error(ft.)
1 0.20 0.47 0.14
2 0.34 0.90 0.35
3 0.42 1.15 0.45
4 0.49 1.24 0.50
5 0.07 0.21 0.02
6 0.13 0.32 0.26
7 0.14 0.24 0.22
8 0.52 1.32 0.56
9 0.04 0.09 0.07
10 0.25 0.63 0.22
Average(ft.) 0.26 0.28 0.66
Average(%) 0.65% 0.56% 0.66%
Table5.MarginofErrorWithoutGCPs
0
1
2
3
4
5
0 50 100 150 200 250 300 350 400
MeasurementError(ft)
FlightAltitude
P4PMeasurementErrorvsFlightAltitude
M1Error
M2Error
M3Error
-12-
DroneDeploy|1045BryantStreet,Suite300|SanFrancisco,CA94103|dronedeploy.com
VisualizingMapResolutionbyAltitudeTherewasasignificantdifferenceinmapresolutionwhenflyingathigheraltitudes.ThiseffectofaltitudeonmapresolutionmadeitdifficulttomarkthecenterofeachgroundcontrolpointwhenprocessingmapdatainDroneDeploy.ForanexampleoftheeffectaltitudehasonimageresolutionseeFigure7.
Figure7.MapResolutionVisualizedbyAltitudeGroundControlPointsImprovedMappingAccuracy10xAcrossthe9mapsprocessedwithGCPs,theaveragemeasurementerrorwasreducedto0.04ft.(0.5inches).Unlikethenon-GCPmaps,theaverageerrorwasnotdependentonthemagnitudeofthecontrollength.The100ft.controllength,whichalwaysproducedthelargestmarginoferrorwhenmeasuredonthenon-GCPmaps,wasthemostaccuratewithanaverageerrorof0.01ft.(0.12inches.Thesesmallerrorswerealsoincrediblyconsistentandhadverylittlevariance.Theaveragestandarddeviation(𝜎")ofmeasurementerrorsonGCPmapswas𝜎" = 0.02,whichwas10xsmallerthannon-GCPmapsasshowninChart2.
-13-
DroneDeploy|1045BryantStreet,Suite300|SanFrancisco,CA94103|dronedeploy.com
Chart2.Phantom4ProMapMeasurementVariance
Conclusion UseHighResolutionCamerasforIncreasedMeasurementAccuracyDroneDeploy’sdatashowsthatusingacamerawithatleast20MPreducestheaveragemarginoferrorinlinearmeasurementsto0.64%.ThedatacollectedusingtheDJIPhantom4Proand20MPcamerawithmechanicalshutterprovidedthemostaccuratedataandmeasurementresultscomparedwithlowerresolutioncamerassuchasthoseusedwiththeDJIMavicProorInspireSeries.DroneDeploysuggestscustomersseekingtomakethemostaccuratemapsformeasuringdistancespoint-to-pointusethePhantom4Pro. FlyMissionsatLowAltitudeforHigherMeasurementAccuracyDroneDeploy’sdatasuggeststhatcreatingflightplanswithlowerelevationscanreducethemarginoferrorsignificantly.Exactreductionsinerroraredependentuponthecameraused.SeeTable4foracomparisonofthisreductioninboth12MPand20MPDJIcameras.DroneDeployrecommendsthatpilotscreateflightplansthatbalancetheneedsofthemissionagainsttheflightsettings.Flylower,withhigheroverlapandsidelap,ifoneintendstomakeaccuratelinearmeasurementsusingthedrone-generatedorthomosaicmap.ThiswillofcourseresultinalargerimagepayloadtobeprocessedwithintheDroneDeploymapengine.Thisincreaseinimageswillalsohaveanincreaseonuploadandprocessingtime.
-14-
DroneDeploy|1045BryantStreet,Suite300|SanFrancisco,CA94103|dronedeploy.com
GroundControlPointsProducetheMostConsistent,AccurateResultsDroneDeploy’sstudyshowsthattherecanbeawiderangeoferrorpresentacrossdatasetscollectedusingDJIdronemodelsandcameras.Theaveragelinearmeasurementmarginoferrorwithouttheuseofgroundcontrolpointswas1.1%ofthecontrollength,however,individualerrorvaluesfellanywhereintherangeof0.01ft.to6.99ft.Ifaprojectrequireshighlyconsistentratesofaccuracy,orshouldmission-criticaldeterminationsneedtobemadefromthedataset,thengroundcontrolpointsshouldbeused.Thisstudyfoundthatprocessingwithgroundcontrolpointsreducedtheaveragemeasurementerrorto0.5inches,anear10ximprovement,forallthecontrollengths.ThemeasurementerrorsforGCPmapsalsoshowedasimilar10xreductioninthestandarddeviationoftheerror,meaningnotonlyisGCPdatamoreaccurate,itismoreconsistentandreliableforapplicationsthatrequirehigheraccuracy.AltitudeMattersWhenMappingwithGroundControlPointsAspreviouslymentioned,imageresolutionwillbeaffectedbyaltitude.ThiswillmakeitdifficulttomarkthecenterofyourgroundcontrolpointmarkerwhenprocessingthemapinDroneDeploy.Accuracywillnotmatterifyouareunabletopreciselyidentifythecenterofyourtarget.PilotsshouldbesuretotakethisintoconsiderationandadjustflightaltitudetoachieveadesiredresolutionthatwillallowthemtoproperlyandconfidentlyidentifytheGCPtargetwithintheirprocesseddronemap.
-15-
DroneDeploy|1045BryantStreet,Suite300|SanFrancisco,CA94103|dronedeploy.com
AdditionalDroneDeployResourcesAGuidetoUsingGroundControlPointswithDroneMappingSoftware
https://blog.dronedeploy.com/what-are-ground-control-points-gcps-and-how-do-i-use-them-4f4c3771fd0bDecidingIfYourDroneMappingProjectNeedsGCPs
https://blog.dronedeploy.com/when-to-use-ground-control-points-2d404d9f5b15CapturingGroundControlPoints(GCPs)
https://support.dronedeploy.com/docs/working-gcp-step-by-stepGCPRequestChecklist
https://support.dronedeploy.com/docs/gcp-request-checklistDroneDeployIn-BrowserGCPTaggingWorkflow
https://support.dronedeploy.com/docs/in-browser-gcp-taggingAcknowledgementsDroneDeploywouldliketothankAlliantUAVforsupportingthisstudybylendingustheEmlidReachRS.ForUSordersoftheEmlidReachRSpleasevisit:https://alliantuav.com/product/reach-rs/DroneDeploywouldalsoliketothankTrimbleforincludingusintheCatalystbetaprogram.FormoreinformationabouttheTrimbleCatalystvisit:https://catalyst.trimble.com/
-16-
DroneDeploy|1045BryantStreet,Suite300|SanFrancisco,CA94103|dronedeploy.com
AppendixViewCompleteDatasetandExploreCompletedMapsandMeasurements:https://www.dropbox.com/s/wqqxhe6od798u4j/Linear%20Measurement%20Accuracy%20of%20DJI%20Drone%20Platforms%20and%20Cloud-Based%20Photogrammetry%20Appendix%20A%2C%20Dataset.xlsx?dl=0
top related