8/6/2019 Counting Large Number of Events in Small Registers

1/3

I t , ; hou ld a l so be no ted tha t ove rpack ing i s a l l owedin the above desc r ibed op t ima l h i s togram ma tch ing p rob -l em. I f ove rpack ing is no t a l l owed and the mono tonep r o p e l t y i s d r o p p e d , t h e n t h e p r o b l e m b e c o m e s t h ec l a ss i ca l b in -pack ing p rob lem [4] which i s kno wn ' t o beN P - c o m p l e t e .

S u b s e q u e n t t o t h i s p a p e r , C h o w a n d K o u f o u n d ad y n a m i c p r o g r a m m i n g a l g o r i th m f o r o p t i m a l h i s t o g r a mmatch ing which a l so has t ime complex i ty O(kl x k2)[1].

Acknowledgments. T h e a u t h o r s a r e i n d e b t e d t o G .M a n a c h e r f o r h i s m a n y h e l p f u l s u g g e s t i o n s a n d c o m -ment s , and to t he re fe ree who p o in t ed ou t an e r ro r i n anea r l i e r ve r s ion o f t h i s pape r .Received June 1977; revised Apri l 1978References1. Chow, W.M., and Kou, L.T. Matchin g two digi ta l pic tures. IBMRes. Rep. RC6870, IBM T.J . W atson Res . Ctr . , Yorktown H eights ,N.Y., Nov. 1977.2. Karp, R.M. Reducibi l i ty amon g combina tor ia l problems. InComplexity of Computer Computations,R.E. Mil ler and J .W.Thatcher , Eds. , Plen um Press , New York, 1972, pp. 85-103.3. Rosen feld, A., and Kak, A.C. Digital Pic ture Processing.Acad emic Press, New Y ork, 1976, pp. 173-175.4. Yao, A. Con crete com put atio nal complexity . Ph.D. Diss., U. ofI l l inois a t Urbana-C hamp aign, 1975.

P r o g r a m m i n gT e c h n i q u e s S.L . Graham, R .L . Rives tEdi to r sCounting LargeNumbers of Events inSmall RegistersRobert MorrisBell Laboratories, Murray Hill, N.J.

I t i s poss ib l e t o use a sma l l coun te r t o kee pa p p r o x i m a t e counts of l a r g e n u m b e r s . T h e resultinge x p e c t e d e r r o r c a n b e r a t h e r p r e c i s e l y co n t r o l le d . A ne x a m p l e is given in which 8-b i t coun te r s (by te s ) a reused t o keep t r ac k o f a s many a s 130 ,000 even t s wi th are l a t i ve e r ro r which is subs t an t i a l l y i ndepe nden t o f t henumb er n o f even t s . Th i s r e l a ti ve e r ro r can be expec t e dto be 24 pe rcen t o r l e ss 95 pe rcen t o f t he t ime ( i . e. o =n/8). The t echn iques cou ld be used to advan tage inm u l t i c h an n e l c o u n t i n g h a r d w a r e o r software used fort h e m o n i t o r i n g o f e x p e r i m e n t s o r p r o c e s s e s .

Key W o r d s a n d Phrases: countingCR Ca tegor i e s : 5 .11

84O

A Counting ProblemAn n-b i t r eg is t e r can o rd in a r i l y on ly be used to coun tup to 2 n - I . I r an in to a p rog ramm ing s i t ua t ion tha t

requ i red us ing a l a rge nu mb er o f coun te r s t o keep t r acko f t h e n u m b e r o f o c c u r r e n ce s o f m a n y d i f f e re n t e v en t s .The coun te r s were 8 -b i t by t e s and because o f t he l imi t eda m o u n t o f s t or a g e a v a il a b l e o n t h e m a c h i n e b e i n g u s e d ,i t was no t poss ib l e t o use 16-b i t coun te r s . Us ing ani n t e r m e d i a t e s i z e c o u n t e r o n a b y t e - o r i e n t e d m a c h i n ew o u l d h a v e c o n s i d e r a b l y i n c r e a s e d b o t h t h e c o m p l e x i t ya n d r u n n i n g t i m e o f t h e p r o g r a m .

T h e r e s u lt i n g li m i t a ti o n o f t h e m a x i m u m c o u n t t o255 led to inaccuracies in the resul ts , s ince the mostc o m m o n e v e n t s w e r e r e c o r d e d a s h a v i n g o c c u r r e d 2 5 5t i m es w h e n i n f ac t s o m e o f t h e m w e r e m u c h m o r e

Permiss ion to copy without fee a l l or par t of this mater ia l isgranted pro vided tha t the copies are not made or dis tr ib uted for direc tcommercia l advantage , the ACM copyright notice and the t i t le of thepublica t ion and i ts da te appear , a nd notice is given tha t copying is bypermiss ion of the Associa t ion for Computing Machinery. To copyotherwise , or to republish, requires a fee and/or specif ic permiss ion.Autho r 's address: Bell Labora tor ies , Murray Hil l , NJ 07974. 1978 ACM 0001-0782/78/1000 -0840 $00.75Com m unic a t ions Oc tobe r 1978of Volume 21the ACM Num be r 10

8/6/2019 Counting Large Number of Events in Small Registers

2/3

f r equen t . I looked in many d i r ec t ions fo r a so lu t ion tothe p rob lem wi th the fo l lowing cons t r a in t s in mind .R u n n i n g t i m e w a s i m p o r t a n t b e c a u s e t h e p r o g r a m w a sa l r eady pa in fu l ly s low . There was no s ign i f ican t add i -t iona l space ava i lab le in the mach ine . On the o the r hand ,p rec ise coun ts o f the even ts were no t ne cessary s ince thep rocess ing was s ta t i s t ica l in na tu re and a r easonab lemarg in o f e r ro r was to le r ab le .

w h e r e n i s t h e n u m b e r o f e v e nt s t h a t h a v e o c c u r re d . Z e r ova lue co r r esponds to ze ro even ts .

Suppose tha t in the mids t o f coun t ing , we hav e theva lue v s to r ed in the r eg is te r . Wh ene ver we ge t ano thereven t , we a t tem p t to mod i fy the con ten ts o f the r eg is te rin the mos t app rop r ia te way . A l l we have i s the va lue v .A s f a r a s w e k n o w , t h e b e s t e s t i m a t e o f t h e n u m b e r o feven ts so f a r i sn~ = e ~ -- 1

A S imp le So lu t ionT h e m o s t o b v i o u s w a y t o c o u n t m o r e t h a n 2 5 5 e v e n t s

in an 8 -b i t r eg is te r i s to coun t on ly every o ther even t .T h i s c a n b e d o n e w i t h a m o d e s t a m o u n t o f e r r o r b ys imp ly f l ipp ing a co in a t eve ry even t to dec ide whe thero r no t to make the coun t . No t on ly i s the expec ted e r ro rsmal l , bu t i t can be p r ec ise ly descr ibed . I n par t icu la r , i fthe num be r o f even ts tha t hav e occu r r ed i s n , then theexpec ted va lue fo r the va lue v in the coun t e r i s n / 2 a n dthe s tandard dev ia t ion i s

so tha t b y the t ime 400 even ts have occu r r ed , v = 200and o = 10. One can expec t tha t 95 percen t o f the t ime ,t h e n u m b e r o f e v e nt s e s t i m a t e d f r o m 2 v i s w i t h i n 4 0 o fthe ac tua l coun t , an e r ro r o f 10 percen t .

T h i s a p p r o a c h c a n b e e x t e n d e d i n t h e o b v i o u s w a yt o c o u n t y e t l a r g e r n u m b e r s o f e v e n ts w i t h c o r r e s p o n d -ing ly incr eased expec ta t ions o f e r ro r by s imp ly u s ing ana p p r o p r i a t e l y w e i g h t e d c o i n . T h i s a p p e a l i n g a p p r o a c hd i d n o t s o l v e m y p r o b l e m b e c a u s e e v e n t h o u g h t h eabso lu te e r ro r was r e la t ive ly smal l , the r e la t ive e r ro r wasin to le r ab ly h igh fo r smal l coun ts . I n f ac t i f one even t hadoccu r r ed , I was guaran teed a 100 percen t e r ro r .A mixed app roach i s poss ib le and overcomes th i sp r o b l e m b y k e e p i n g a c t u a l c o u n t s u p t o s o m e p r e s e tn u m b e r a n d u s i n g t h e c oi n - f li p p i n g a p p r o a c h a b o v e t h a tpo in t . I t i s a genera l iza t ion o f th is app r oac h tha t l ed tothe success fu l so lu t ion to my p rob lem.

A G enera l i za t i onSuppose tha t in s tead o f t r y ing to keep t r ack o f the

n u m b e r n o f e ve n t s o r o f s o m e c o n s t a n t m u l t i p l e o f n,tha t we keep in the r eg is te r the logar i thm o f the num be ro f e v e n ts a n d d e v i se s o m e s o r t o f a p p r o a c h a s w a s s e tou t above . The n i f the va lue (wh ich i s a logar i thm)su f f e r s f rom a g iven abso lu te e r ro r , then the number o feven ts wh ich th i s logar i thm r e f lec t s i s a f f ec ted on ly by ar e la t ive e r ro r. Th i s i s wha t i s genera l ly des i r ed.

The s imp les t app roach tha t u ses th i s idea i s to p ro -ceed as fo l lows: The v alue v s tored in the regis ter isimag ined to r ep resen tv (n) = log( l + n)

so tha t the num ber o f even ts inc lud ing the cu r r en t oneis d . The va lue tha t we wou ld l ike to r e tu rn to theregis ter isv ' = log( l + e ~)bu t th i s is no t in genera l an in teger and we canno t ju s tt r u n c a t e o r r o u n d t h e q u a n t i t y f o r f e ar o f a c c u m u l a t i n gser ious e r ro r . I n s tead we le t v' = k + f where k is anin tege r and 0 ~< f < l , and r eturn ei the r the value k orthe val ue k + 1 to the regis ter as the new value o f v withapp rop r ia te ly chosen p robab i l i t i e s so tha t the expec tedvalue of v is correct .

The co r r ec t th ing to do i s to compu teI/A = n~-i -- n~.In every case o f in te r est , A wi ll be be tween 0 and 1 andso we can igno re the poss ib i l i ty o f an in te ger par t . W et h e n o bt a i n a r a n d o m n u m b e r r fr o m a r a n d o m n u m b e rgenera to r u n i fo rml y d is t r ibu ted in the in te rva l 0 ~< r r , s e t v = v + 1i f A ~ < r , s e t v = v .I t i s easy to p rove tha t we have no t by th i s p rocedu red is tu rbed the expec ted va lue o f the con ten ts o f ther eg is te r . To p rove th i s , observe tha t the r andom p roce-du re subs t i tu tes fo r the co r r ec t va lue v' e i therv w i t h p r o b a b i l i t y 1 - A , o rv + 1 with prob abi l i ty A.The n the expec ted va lue o f n r ep resen ted by the con ten tsof v is equal toAn~-m + (l -- A)n~ = A(n~+l - n~) + n,

e ~ + 1 - l - e " + 1= + n~e ~ ( e - l )= n o + 1 .

Af ter one even t has occu r r ed , the r eg is te r con ta in sthe va lue 1 w i th p roba b i l i ty . 59, o r 0 w i th p robab i l i t y.41 . When we come to in te rp re t th i s va lue we wou ldc o n c l u d e t h a t t h e n u m b e r o f e v e n t s w a s1.7 59 perce nt of the t ime, and

0 41 perce nt of the t ime.The expec ted va lue in the r eg is te r i s co r r ec t , bu t theactual value is way off .

841 Communications October 1978of Volume 21the ACM Numbe r 10

8/6/2019 Counting Large Number of Events in Small Registers

3/3

W e c a n j i g g le t h e p a r a m e t e r s o f t h e m e t h o d s o t ha ta coun t o f one r esu l t s in a r eg is te r va lue o f one and thenthe r an dom round ing p rocedu re has no e f f ec t on the fi r s tcoun t . The func t ionv = log(n + 1)/log (2)has ju s t the p ro per ty we wa n t . Th is fo rmu la i s o f cou r sei n d e p e n d e n t o f t he b a s e o f t he l o g a r i t h m s .

A General SolutionT h e c l as s o f f u n c t i on s t h a t I h a v e u s e d a n d a n a l y z e d

are the func t ionsu(n) = log O + n / a ) / l o g ( 1 + l / a )w h e r e t h e p a r a m e t e r a co n t r ol s b o t h th e m a x i m u m c o u n ttha t can be he ld in the r eg is te r and the expec ted e r ro r .The cons tan t log O + 1 / a ) i n t h e d e n o m i n a t o r s e r v e sonly to force n = l to corr esp ond to u = 1 so that th er a n d o m p r o c e d u r e s h a v e n o e f f ec t on t h e f i rs t c o u n t a n dcoun ts o f 0 and 1 a r e r ep resen te d exac t ly . I t i s in th i sway tha t good r e la t ive accu racy i s p r eserved fo r smal lc o u n t s . T h e m a x i m u m v a l u e n t h a t c a n b e r e p r e s e n t e du s i n g t he p a r a m e t e r a c a n b e c a l c u l a t e d f r o m t h e i n v e r t e df o r m u l an, = a((1 + 1 / a f - 1).The expec ted e r ro r in the es t im ated va lue o f n a f te r nc o u n t s c a n b e c a l c u l at e d f r o m t h e f o r m u l ao z = n( n - 1)/2a.T h i s f o r m u l a c a n b e p r o v e d b y i n d u c t i on o n n .

L e t u s i n s p ec t t h e p e r f o r m a n c e o f th i s m e t h o d u s i n gthe parameter a = 30 . The la rges t va lue tha t can berep resen ted in an 8 -b i t coun te r i s abou t 130 ,000 . Thes t a n d a r d d e v i a t i o n o is a p p r o x i m a t e l y e q u a l t o n / 8 w h i c himp l ies tha t the r e la t ive e r ro r is near ly ind epen den t o f nand tha t 95 percen t o f the t im e the r e la t ive e r ro r w i l l beless than 24 percen t . Larg er va lues o f a w i l l l ead tos m a l l e r m a x i m u m c o u n t s a n d , o f c o u rs e , to s m a l l e rr e la t ive e r ro r .

T h e r e i s n o n e e d t o c o m p u t e a n y o f t he l o g a r i t h m s o rpower s du r ing the coun t ing p rocess . A tab le con ta in ingt h e 2 55 v a l u e s o f A c a n b e p r e c o m p u t e d b y t h e f o r m u l aA y = ( a / ( a + 1)) and accessed when a new coun t i sm a d e . T h e r a n d o m n u m b e r g e n e r a t or c a n b e o f t h es i m p l e s t s o r t a n d n o g r e a t d e m a n d s a r e m a d e o n i t sp roper t ies .T h e d i s t r ib u t i o n o f e r ro r s i s s o m e w h a t a s y m m e t r i cfo r smal l coun ts , bu t as n become s la rger, the d i s t r ibu t ionc lo se ly app rox imates the no rmal d i s t r ibu t ion . I n theexample above where a - - 30 , the no rmal e r ro r cu rveg ives a u sefu l es t imate o f the e r ro r d i s t r ibu t ion fo r coun tsg rea te r than abou t 20 .Received June 1975; revised Decemb er 197784 2

P r o g r a m m i n gT e c h n i q u e s

S . L . G r a h a m , R . L . R i v e s tEd i to r sAn Analysis ofAlgorithms for theDutch National FlagProblemCol in L. McMasterUnivers i ty of C al i forn ia

Solutions to the Dutch National Flag Problem havebeen given by Dijkstra [1] and M eyer [3]. Dijkstrastarts with a simple program and arrives at an improvedprogram by refinement. B oth o f the algorithms given byDijkstra are shown to have an expected number ofswaps which is 2N + 0(1) and that these values differ atmost by 1 of a swap and asymptotically by of a swap.The algorithm of M eyer is shown to have exp ectedswap complexity ~N.Key W ords and Phrases: algorithmic analysis,Dutch National Flag Problem, refinement, structuredprogramming

CR Categories: 4.0, 5.24, 5.25, 5.3

IntroductionDijks t r a [1 ] has def ined a p rob lem wh ich he ca l l s the

P r o b l e m o f t he D u t c h N a t i o n a l F l a g . I t m a y b e s t a t ed a sf o ll o w s. T h e r e i s a r o w o f N b u c k e t s n u m b e r e d f r o m 1t o N . T h e b u c k e t s a r e a r r a n g e d i n n u m e r i c a l o r d e r w i t hbucke t 1 on the le f t and bucke t N on the r igh t . Eachb u c k e t c o n t a i n s e x a c t l y o n e p e b b l e a n d e a c h p e b b l e i sPermiss ion to copy without fee a l l or par t of this mater ia l isgranted pro vided tha t the copies are not mad e or dis tr ibuted for direc tcommercia l advantage , the ACM copyright notice and the t i t le of thepublica t ion an d i ts da te appear , and n otice is given tha t copying is bypermiss ion of the Associa t ion for Computing Machinery. To copyotherwise , or to republish, requires a fee and/or specif ic permiss ion.Research sponsored by Joint Services Elec tronics Program GrantF 44620-76-C-0100 a nd Na t iona l Sc ie nce F ounda t ion Gra n t MCS 76-15036.Auth or 's address : Departme nt of Elec tr ica l Engineerin g and Co m-puter Sciences, Univers i ty o f California , Berkeley, CA 94720. 1978 ACM 0001-0782/78/0000-0842 $00.75Comm unicat ion s October 1978of Volum e 21t h e A C M N u m b e r 1 0