how to write a good papergernot/talk-howto-paper.pdf · 2018. 10. 7. · 1. the paper has a (one!)...
TRANSCRIPT
www.data61.csiro.au
HowtoWriteaGoodPaper
GernotHeiser|[email protected]|@GernotHeiser
TSBootcampMay’18
...orwhatIwishmysupervisorhadtaughtme
TSBootcampMay'182|
RejectionIsLife
• My2013stats(mysecond-bestyearever!):– 11accepts:– 6conferences:EuroSys,SIGMOD,SOSP,OOPSLA,2*RTAS– 4workshops:HotOS,APSys,PLOS,HotPower,– 1journal:TOCS(plusTODSinvite)
– 8rejects:2*Usenix,PLDI,2*RTSS,APSys,EMSOFT,RTAS• My2017stats(abadyear):– 4accepts:– 1conference:EuroSys(paperrejected5times!)– 2workshops:PLOS,APSys– 1magazine(invited):IEEEDesign&Test
– 7rejects:– UsenixSecurity,IEEES&P×2,RTAS,ASPLOS,SOSP×2
3| TSBootcampMay'18
Qualifications?
IEEEAuthoringWorkshop,UNSW,July'18©GernotHeiser2018,releasedunderCCAttributionLicense4|
• Servedonalltop-tierconferenceTPCsinmyfield• atleastonetop-tierPCperyear• Presently• associateeditorofIEEETransactionsonComputers• EBmemberofACMCommunicationsResearchHighlights
WaystoAvoidRejection
• Safeway:Aimlow:• 2nd/3rd-tiervenuesareeasy• guaranteedimpact-free
IEEEAuthoringWorkshop,UNSW,July'18©GernotHeiser2018,releasedunderCCAttributionLicense5|
THIS ADVICE IS PROVIDED ``AS IS'' AND WITHOUT ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE.
Onlywaytobuildreputation!
• Hardway:Writeanexcellentpaperwithasignificantcontribution
JournalsvsConferences
• I’minComputerScience,sub-disciplineOperatingSystems• 90%oftopOSpublicationsareinconferences• othersub-disciplinesaredifferent:CStheoryisabout50%journals• CStopconferencereviewingmuchtougherthanjournals• Non-CSconferencesaremorelightweight,journalsdominate
IEEEAuthoringWorkshop,UNSW,July'18©GernotHeiser2018,releasedunderCCAttributionLicense6|
ComputerScienceEspeciallysystems
Other
Complete,significantwork
Top-tierconference Transactions
Lesssignificant,recycledrejects
Secondaryconference,journal
Otherjournal
Earlywork,positionpapers
Workshop Conference
Whatis“Systems”?
(Overly?)simplifiedviewofComputerscience:theory+systems• Theoristsbuildtheories,models• oftengetawaywiththeoriesnotgoodforanything• Systemsfolksbuildstuff• don’tgetawaywithworknotgoodforanything!
Examplesof“systems”work:• operatingsystems• networksystems/distributedsystems• databasesystems• programmingsystems(PLimplementation)• machine-learningsystems• …
Mycommunity,i.e.Iunderstandhowit
ticks
TSBootcampMay'187|
RulesofWriting
Rule1:ReviewersarePotLuck
• …evenattopconferences• evengoodpapersgetrejected,sometimesforthewrongreasons• …butmosttimesrejectionisyourfault!
• Reviewers’topreasonsforrejection• I’mnotconvincedyou’resolvingarealproblem• I’mnotconvincedyou’resolvingtheproblem• Idon’tunderstand–yourpaperistoobadlywritten• Insufficientcontributionfor{SOSP,OSDI,EuroSys…}
• PaperswithoutaPC“champion”haveahardstand• Makesurethere’ssomethingwhichatleastonereviewerwillthinkcool• Purelyincrementalworkwillhaveahardstandattopvenues
TSBootcampMay'189|
Rule2:APaperHasaStory
1. Thepaperhasa(one!)mainmessage• Understandclearlywhatthemessageis• Makesurethatthereadergetsit• Makesureit’saninterestingone
2. Apaperhasanarrative• Itstartsfromzeroandthenworksontransmittingthemessage• Everythingyouwritemustsupportthemessage• Maintainreaderstate!
– beconsciousofwhatthereaderknows/remembers
TSBootcampMay'1810|
Rule3:LimitedSpace:TheTwo“C”s
• Beclear(atalllevels)• everysentence,paragraph,sectionhasaclearpurpose• thepurposeisclearlycommunicated• theoverallmessageisconsistent• Beconcise(briefbutcomplete)• don’twaffle!!!(Use“Jay’sruleofthumb”)• beprecise• makesureit’sreadable,lucid,enjoyable
But:• maintainreaderstate:– definebeforeuse– beawareofwhatthereaderhaslearned– recall/remindifnecessary
• Makesureit’sself-containedTSBootcampMay'1811|
Rule4:PresentationMatters
TSBootcampMay'1812|
Topconferencestendtoaccepttwokindsofpapers
1. excellentworkthatispresentedwell
2. averageworkthatispresentedwell
Thebestworkisuselessifyoucan’tconvincethereviewers• reviewersarebusy,mayhavetoreview30papers• they’lllookforreasonstoreject–don’tgivethemany!
Rule4:PresentationMatters–PaperEngineering
Importantbits:• Introduction:selltheidea,thesignificanceandtheapproach
• buildtension,makereaderinterested
• convincingargumentation
• top-down,notbottom-up
• maintainreaderstate
• convincingevaluation
• thoroughandhonest
• stateassumption/limitationshonestly
TSBootcampMay'1813|
PaperStructureandStyle
Introduction:MostImportantPartofthePaper!TheOverture:• Explaintheproblemyou’resolving,whyit’saproblem• Outlineyourapproach• Indicateresults/outcomes• Statecontributions• “Paperroadmap”isawasteofspace
Generalhintsforintro:• Capturethereader’sinterest:sellyouridea• Beconcise:Staywithinaboutonepage!• Makesurethepaperdeliverswhatyoupromise• Reviewerskillfor“baitandswitch”
TSBootcampMay'1815|
OtherParts
• Background:setthesceneinmoredetail• citerelatedworkasneeded,don’tdiscussmorethannecessary• Examples!!!!• Describeproblemindetail• Explainsolutionindetail• behonestandforthcomingwithlimitationsandassumptions• Evaluation:forsystemsworkoftenlargestpart• Relatedwork• Conclusions• Abstract• usedtosteertotherightreviewers• What,Why,Achievement,Implication,onesentenceeach• IMPORTANT:Redoforcamera-ready!
TSBootcampMay'1816|
Evaluation
• Showthatyoursolutionactuallyworks• Progressive:significantimprovementsinimportantsituations• Conservative:no(significant)degradationelsewhereNeedboth!
TSBootcampMay'1817|
BenchmarkingCrimes(Selection)
1. SelectiveBenchmarking–cherrypicking2. Onlymicro-benchmarks3. Throughputdegradation=overhead4. Creativeoverheadaccounting5. Improperbaseline,onlyrelativefigures,compareagainstself6. Noindicationofsignificance
Fulllist:http://gernot-heiser.org/benchmarking-crimes.html
TSBootcampMay'1818|
StyleandForm• Writeinengagingstyle,leadreaderthoughthepaper• avoidbottom-upstructure,presentideastop-down• followstylerules• Useactivevoice!!!!…andpresenttense• Avoidbuzzwords(“novel”,“mobilesocialpost-quantumfogcomputing”)• Bemindfulofreader’sbrainstate(whichislossy)• maintainreaderstate• don’tassumeeveryreviewerisexpertinyournarrowarea• butdon’tthinkyoucanhidestufffromreviewers!• Followformattingrules• don’tplaywithmargin,baselineskipetc• don’tusemicroscopicfonts,>40yoldshaveproblemswith<8ptfont• Spell-check,proof-read,proof-read• getnativespeakertoproof-readifyouaren’t• getoutsidertoreadit–greatwaytospotholesbeforeit’stoolate!TSBootcampMay'1819|
Mechanics• Userevisioncontrol• especially(butnotonly)whenit’sajointpaper
• Don’tuseMSWord• doesn’tintegratewellwithrevisioncontrol• requirescoarse-grainlocking• referencesarepainful,formulaeevenmoreso• MSRpeopleuseLaTeX,soshouldyou!
• UseBibTeX• …butuseitcorrectly(egcapitalisationintitles)
• Usescriptabletools(egGNUplot)forgraphingresults• Resultschangefrequentlyandatthelastminute• Beingabletorunfromcommandlineisessential
TSBootcampMay'1820|
Summary
• Clearstatementofproblem• WhywouldIcare?• Convincingsolution,compellingargument• Thoroughevaluation,noBMcrimes• Lucidwriting,maintainingreaderstate
TSBootcampMay'1821|
FurtherReading
Writingsystemspapers:• Levin&Redell:Anevaluationofthe9thSOSPsubmissions,orHow(andhownot)towriteagoodsystemspaper
• SimonPeytonJones(MSRC):Howtowriteagreatresearchpaper• http://research.microsoft.com/en-us/um/people/simonpj/papers/giving-a-talk/giving-a-talk-slides.pdf
• Mypaper/thesiswritingguide• http://gernot-heiser.org/style-guide.html
Generalwriting/styleetc(recommendedbysystemsfolks):• Zobel:Writingforcomputerscience,Springer• Strunk&White:Theelementsofstyle,Allyn&Bacon• Dupré:Bugsinwriting:Aguidetodebuggingyourprose,Addison-Wesley
TSBootcampMay'1822|
www.csiro.au
Thankyou
TSBootcampMay'1823|