24 ECTI TRANSACTIONS ON ELECTRICAL ENG., ELECTRONICS, AND COMMUNICATIONS VOL.12, NO.1 February 2014

Factorial Hidden Markov Model analysis ofRandom Telegraph Noise in Resistive Random

Access Memories

Francesco Maria Puglisi∗1 and Paolo Pavan∗ , Non-members

ABSTRACT

This paper presents a new technique to analyze thecharacteristics of multi-level random telegraph noise(RTN). RTN is de�ned as an abrupt switching of ei-ther the current or the voltage between discrete valuesas a result of trapping/de-trapping activity. RTN sig-nal properties are deduced exploiting a factorial hid-den Markov model (FHMM). The proposed methodconsiders the measured multi-level RTN as a super-position of many two-levels RTNs, each representedby a Markov chain and associated to a single trap,and it is used to retrieve the statistical properties ofeach chain. These properties (i.e. dwell times andamplitude) are directly related to physical propertiesof each trap.

Keywords: RTN, Multi-level, FHMM, Trapping,Noise.

1. INTRODUCTION

Random telegraph noise (RTN) is usually foundin metal-oxide-semiconductor �eld-e�ect transitors(MOSFETs) and in other novel devices (e.g. Resis-tive Random Access Memeories, RRAM [1-3]) as anabrupt and random change of either the voltage or thecurrent between discrete levels. This can result in un-predictable deviation of key parameters (e.g. thresh-old voltage of MOSFETs, read current in RRAMs)from their expected values. Currently, RTN is be-coming a challenging issue limiting the full industrialexploitation of RRAM concepts and their reliabilitysince its e�ects are expected to be even more severe innanoscale devices. Even though the physical mecha-nisms responsible for RTN have not been completelyassessed, it is commonly accepted that it is the resultof capture and emission processes of charge carriersin/from defect centers acting as traps [4]. Fig. 1shows our interpretation of the mechanism leading toboth two-levels (a) and multi-level (b) RTN in metal-oxide-based RRAMs in High Resistance State (HRS)along with experimental time series [5,6]. Our pre-vious works [5,6] exploited the color-coded time-lagplots to investigate the nature of RTN in RRAMs,

Manuscript received on January 15, 2014 ; revised on Febru-ary 2, 2014.

∗The authors are with with DIEF - Univer-sità di Modena e Reggio Emilia, Italy, E-mail:francescomaria.puglisi@unimore.it1

showing that multi-level RTN can be seen as a super-position of many two-levels RTNs. Formerly, we usedhidden Markov model [7] (HMM) to investigate RTN.However, HMM can be used to extract the discretelevels in the current but cannot be used directly toextract the amplitude of each two-levels �uctuationin multi-level RTN.

In this paper, we propose a more re�ned implemen-tation of the HMM which is best suited to solve forthe statistical properties of multi-level RTN causedby multiple traps. Retrieving traps parameters is ofutmost importance to gain a deeper understanding ofthe phsyical mechanisms leading to RTN since theyare strictly related to the physical properties of thedefect centers, such as their positions and energiesin the oxide layer and their relaxation energy (takinginto account the structural lattice relaxation occuringduring the trapping and detrapping of charge carri-ers) [4]. Traps parameters are estimated using a fac-torial hidden Markov model [8,9] (FHMM) approach:the the proposed method is self-consistent (the num-ber of active traps is automatically determined) andits implementation can be parallelized, leading to bet-ter performances with respect to other methods suchas Markov chain Monte Carlo-based techniques [10].This paper is organized as follows: the mathematicaldescription is given in Section 2, underlining the limi-tations of HMM approach when analyzing multi-levelRTN and proposing FHMM; in Section 3 we reportresults and discussion. Conclusions follow.

2. STATISTICS BACKGROUND

The capture/emission process of charge carri-ers in/from traps into the barrier, Fig. 1, canbe described by a Hidden Markov Model, i.e. aMarkov (memoryless) process with unobserved (hid-den) states. Whereas in simple Markov models thestate of the system at each instant of time is directlyvisible to the observer, in HMM the output of the sys-tem is directly visible at each instant of time whilethe state of the system is hidden, even though theoutput strictly depends on the state. Each state ischaracterizied by a probability distribution over allthe possible values assumed by the output, statisti-cally linking the sequence of observations (output) tothe sequence of hidden states. Moreover, each stateis associated to a set of transition probabilities (oneper each state) de�ning how likely is for the system,

Factorial Hidden Markov Model analysis of Random Telegraph Noise in Resistive Random Access Memories 25

Fig.1: a) Experimental two-levels RTN and its sim-pli�ed physical mechanism involving one trap only. b)Experimental multi-level RTN and its simpli�ed phys-ical mechanism involving two (many) traps. Blackspots are the charge carriers and the black holes areactive traps. Metal-oxide-based RRAM conduction inHRS is modelled as trap-assisted tunneling via thetraps in the barrier, leading to 2-levels (a) or multi-level RTN (b).

being in a given state at a given instant of time, toswitch to another of the possible states (including thesame state) at the successive instant of time.

2.1 The probabilistic model of HMM

In HMM, a sequence of observations {Y t} t = 1...Tis modeled by specifying a probabilistic relation be-tween the observations and a set of hidden (unknowna priori) states St through a Markov transition struc-ture linking the states. In this framework the state isrepresented by a random variable assuming one out ofN values at each instant of time. The HMM approachrelies on two conditional independence assumptions:

1) St only depends on St−1 (known as the �rst-order Markov or �memory-less� property)

2) Yt is independent of all other observationsY1, ..., Yt−1, Yt+1, ..., YT given St.

The joint probability for the state sequence andobservations can be formalized as:

P (St|yt) = Pinit �T∏

t=2

P (St|St− 1) � P (Yt|St)

Pinit = P (S1) � P (Y1|S1)

(1)

A schematic representation of the HMM is givenin Fig. 2, where the Markov property is evi-denced. According to the formalism used by Rabinerin [7], an HMM is completely de�ned as a 5-tuple(N,M,A,B, π). N is the number of hidden states, S,in the model (i.e. the number of discrete current lev-els to be found in RTN); since observations assumediscrete values, M is de�ned as the number of distinctobservable symbols (i.e. the possible current valuesassumed by RTN). A is an N-by-N matrix de�ningthe transition probabilities among states and B is aN-by-M matrix de�ning the observation probabilityof each observable symbol in each hidden state; π is a

vector de�ning the initial state probability distribu-tion [7]. The inference problem in this model consistsin �nding the most likely set of probability of hid-den states given the observations. This is achievedthrough a maximum likelihood estimate of the HMMparameters given the observations using the forward-backward algorithm [7]. Then the most likely se-quence of hidden states representing the dynamics ofthe observations can be achieved via the �Viterbi� al-gorithm, a dynamic programming paradigm. As aresult, HMM analysis can e�ciently estimate the dis-crete current levels and the best sequence of statesrepresenting RTN data, as shown in Fig. 3(a).

Fig.2: Graphical representation of an HMM. Ateach instant of time t, each output Yt is related onlyto the current state of the Markov chain de�ning themodel.

Fig.3: a) Experimental two-levels RTN and HMM�tting. The hidden levels and most likely state se-quence are correctly retrieved. The distinctive fea-tures of the two-levels RTN (amplitude of the �uc-tuation and average capture/emission times) are ev-idenced. b) Experimental multi-level RTN and HMM�ting. Though succesful in characterizing the hiddenstates and their durations, HMM output is unsu�-cient to achieve a comprehensive characterization ofall the traps leading to the observed noise.

2.2 The limitations of HMM for multi-level

RTN analysis

Even though being e�ective in capturing theMarkov dynamics of RTN, HMM is nevertheless un-suitable to comprehensively characterize multi-levelRTN, although is still valid to fully describe a two-levels RTN (for which N = 2). Indeed, the distinc-tive feature of a two-levels RTN �uctuation, relatedto the physical properties of the associated trap, are

26 ECTI TRANSACTIONS ON ELECTRICAL ENG., ELECTRONICS, AND COMMUNICATIONS VOL.12, NO.1 February 2014

the amplitude of the �uctuation and the average cap-ture/emission times, see Fig. 3(a). While dealingwith a two-levels �uctuation, the output of the HMManalysis is su�cient to extrapolate all the character-istic features of the RTN data: the amplitude of the�uctuation is simply given by the di�erence betweenthe two hidden states while the average capture andemission times can be extracted by averaging the du-ration of each state. In the case ofmulti−level RTN, a superposition of many two-levels RTNs each gen-erated by the contribution of a single trap [5,6], acomplete characterization would be achieved only byde�ning all the distinctive features of every trap con-tributing to the observed RTN. Unfortunately thisresult cannot be achieved with the HMM approach:Fig. 3(b) reports an experimental multi-level RTN(generated by many traps) along with the HMM out-put. Though the HMM analysis is correctly de�ningthe hidden states of the multi-level RTN and theirmost likely sequence, it is generally impossible to sep-arately de�ne the amplitudes of �uctuations and cap-ture/emission times for each single trap contributingto the RTN. This implies that even though the char-acterization of the RTN signal is achieved, it is im-possible to retrieve the distinctive features of eachtrap contributing to the observed noise. In this pa-per we show how this limitation can be overcome byusing a more re�ned HMM-based concept, namely theFHMM.

The FHMM [8] extends the HMM potential byconsidering the hidden state as a collection of K statevariables, instead of a single random variable, eachpotentially assuming one out of N values at each in-stant of time (i.e. K di�erent and parallel Markovchains). This results in a space state having a dimen-sion of NK . If no constraints are applied to the model,it can potentially take into account all the possibleinterdependencies between the K Markov chains, re-sulting in a high computational burden. However, anatural approach consists in assuming that each ofthe K Markov chains evolves independently from theother chains, resulting in a signi�cant reduction ofthe problem complexity. This can be formalized as:

P (St|St−1) =

K∏k=1

P (Skt |Sk

t−1) (2)

This is also the most suitable representation of amulti-level RTN, seen as a superposition of many two-levels RTNs [5,6], each associated to a single trap.This assumption also constraints each Markov chainstate to assume only one out of two values at eachinstant of time (which is N=2). A graphical repre-sentation of the FHMM concept is given in Fig. 4:Smt represents the state of the m − th chain at time

t, while Yt represents the output of the whole pro-cess (i.e. the expected value of the multi-level RTN)at time t. The inference problem is solved by usingthe expectation-maximization algorithm, an iterative

method for �nding maximum likelihood estimates ofparameters in statistical models depending on hiddenvariables. The iteration alternates between an expec-tation step, calculating the expectation of the like-lihood evaluated using the current estimate for themodel parameters, and a maximization step, com-puting the parameters maximizing the expected like-lihood found on the previous step. These estimatescan be used to determine the probability distributionof the hidden variables in the next iteration. This ap-proach allows decomposing the multi-level RTN intoa superposition of two-levels RTNs: since the outputof the FHMM is a collection of two-levels �uctuations,it is now possible to separately retrieve the distinc-tive features of each trap contributing to the observedmulti-level RTN.

Fig.4: Graphical representation of an FHMM. Ateach instant of time t, each output Yt is related tothe superposition of the states of M independent andparallel Markov chains.

2.3 Implementation issues and self-consistency

The implementation of either HMM or FHMM suf-fers from a trade-o� between the computational bur-den and the �tting accuracy. Indeed, a more com-plex model (higher number of hidden states in HMMor higher number of Markov chains in FHMM) re-sults in higher time-to-solution. Regrettably, as inHMM the number of hidden states is an input param-eter for the model, so the number of parallel Markovchains (i.e. the number of traps contributing to theobserved RTN) should be estimated beforehand inFHMM. This theoretically requires an a priori esti-mation of the number of traps contributing to theRTN. However this issue can be solved by feedingthe algorithm a reasonably large number of expectedtraps (though resulting in a more time-consuming al-gorithm): the chains related to traps which are un-necessary to match the input RTN will be character-ized by negligible amplitude of the �uctuation andcan easily be discarded after the analysis. The ad-vantage of the FHMM over HMM is evident even inthis aspect: a too large estimation of the number ofhidden states in HMM can cause the algorithm to beforced to identify more hidden levels than the e�ec-tive number, resulting in an erratic signal characteri-zation. Instead in the FHMM approach, using a large

Factorial Hidden Markov Model analysis of Random Telegraph Noise in Resistive Random Access Memories 27

number of parallel chains is not a�ecting the goodnessof �tting. This results in the FHMM approach to beself-consistent and extremely accurate.

3. RESULTS AND DISCUSSION

The proposed method has been tested using math-ematically generated RTN data simulating the activ-ity of three traps with additive gaussian noise. Thenumber of expected traps was intentionally set to �ve,a reasonably high value, in order to check the al-gorithm capability of automatically determining thenumber of active traps contributing to the observedRTN. Results are summarized in Tab. 1 and plot-ted in Fig. 5. Fig.5 (a) shows the extremely accu-rate matching of the FHMM output and the inputmulti-level RTN. Moreover, input data are correctlyseparated in three two-levels RTNs time series withremarkable accuracy, Fig.5 (b-d). Noticeably, sinceonly three traps were necessary to match the inputsignal, the algorithm assigned negligible amplitudeto two out of �ve chains (see Tab. 1), con�rming itsself-consistency. The algorithm was also successfullyapplied to a real experimental time series, see Fig. 6.Results are summarized in Tab. 2.

Other statistical machine learning methods cansolve this problem using Markov chain Monte Carlo(MCMC) approaches, the simplest of which is Gibbssampling [10]. Although this technique is guaranteedto converge to the real probability distribution of thedata [8,10], the whole space de�ned by unknown vari-ables in the model has to be sampled, resulting inan extreme computational burden as a consequenceof the so-called curse of dimensionality. Moreover,Gibbs sampling technique is intrinsically non paral-lelizable since every step is based on the result ofthe previous one, preventing an e�cient implementa-tion from being possible. Conversely, the FHMM ap-proach takes advantage of parallel computing: sincethe time-to-solution is strictly dependent on the ini-tial guess of the model variables (namely A, B andπ, according to the formalism used by Rabiner it ispossible to run parallel instances of the FHMM al-gorithm on di�erent cores and then choose the resultmaximizing the likelihood. Furthermore, as shownin [8], approximated inference techniques can be usedto speed-up the FHMM routine: the expectation stepof the expectation-maximization algorithm discussedin Section 2.3 can be replaced with either a Gibbssampling approach or a variation inference method atthe cost of lower accuracy. Nevertheless the speed-upgain is strictly dependent on the model complexityand, in our speci�c case, the exact expectation hasbeen found to be the best choice in terms of trade-o�between a good accuracy and a reasonable time-to-solution. [7]), it is possible to run parallel instancesof the FHMM algorithm on di�erent cores and thenchoose the result maximizing the likelihood. Further-more, as shown in [8], approximated inference tech-

niques can be used to speed-up the FHMM routine:the expectation step of the expectation-maximizationalgorithm discussed in Section 2.3 can be replacedwith either a Gibbs sampling approach or a variationinference method at the cost of lower accuracy. Nev-ertheless the speed-up gain is strictly dependent onthe model complexity and, in our speci�c case, the ex-act expectation has been found to be the best choicein terms of trade-o� between a good accuracy and areasonable time-to-solution.

Table 1: FHMM Output for Generated Data.

Trap Amplitude Amplitude AmplitudeNr. (Generated) (FHMM) % of Trap 11 2 1.994 100.00%2 1 1.004 50.35%3 5 4.998 250.65%4 - 0.024 (discarded) 1.20%5 - 0.002 (discarded) 0.10%

Table 2: FHMM Output for Experimental Data.

Trap Amplitude AmplitudeNr. (FHMM) (% of Trap 1)1 1.78× 10−8 100.00%2 9.87× 10−9 55.45%3 4.1× 10−11 (discarded) 0.23%4 3.4× 10−11 (discarded) 0.19%5 7.6× 10−12 (discarded) 0.04%

Fig.5: a) An eight-levels RTN generated by threetraps with superimposed additive gaussian noise (bluecurve) and FHMM �tting (red curve). b, c, d) Ampli-tudes of �uctuations and state sequences for all trapsare easily found and traps characteristics can be in-ferred.

4. CONCLUSIONS

In this paper we proposed the FHMM approachto achieve a full and comprehensive characterizationof multi-level RTN, resulting from the activity ofmultiple traps. HMM limitations in characterizingthe multi-level RTN were underlined and the novelFHMM approach has been used to solve for the sta-tistical properties of each trap contributing to multi-

28 ECTI TRANSACTIONS ON ELECTRICAL ENG., ELECTRONICS, AND COMMUNICATIONS VOL.12, NO.1 February 2014

Fig.6: a) An experimental multi-level RTN and b)FHMM �tting. c, d) Amplitudes of �uctuations andstate sequences for all traps are easily found and trapscharacteristics can be inferred.

level RTN. As a result, a complex RTN has been sepa-rated in multiple two-levels RTNs, allowing inferringthe distinctive features of each of the traps leadingto the observed RTN. This is of crucial relevance forstudying the trap-assisted conduction in novel devicesas the inferred trap properties are directly linked totheir physical properties. This method is compre-hensive and self-consistent, and has been succesfullytested with both experimental and mathematicallygenerated data. Moreover it can take advantage ofparallel computing on distributed cores, resulting ina consistent speed-up.

References

[1] F. M. Puglisi et al., �An empirical model forRRAM resistance in low- and high-resistancestates,� IEEE Electron. Device Lett., vol.34,no.3, pp. 387-389, Mar. 2013.

[2] F. M. Puglisi et al., �A compact model ofhafnium-oxide-based resistive random accessmemory,� Proc. Int. Conf. IC Design Technol-ogy, 2013, pp. 85-88.

[3] D. Veksler et al., �Random telegraph noise(RTN) in scaled RRAM devices,� Proc. IEEEInt. Reliability Physics Symp., 2013, pp.MY.10.1-MY.10.4.

[4] L. Vandelli et al., �A physical model of thetemperature dependence of the current throughSiO2/HfO2 stacks,� IEEE Trans. Electron De-vices, vol. 58, no. 9, pp. 2878-2887, Sept. 2011.

[5] F. M. Puglisi et al., �Random telegraph signalnoise properties of HfOx RRAM in high resis-tive state,� Proc. European Solid-State DeviceResearch Conf., 2012, pp. 274-277.

[6] F. M. Puglisi et al., �RTS noise characterizationof HfOx RRAM in high resistive state,� Solid-State Electron., vol. 84, pp. 160-166, Jun. 2013.

[7] L. R. Rabiner, �A tutorial on hidden Markovmodels and selected applications in speech recog-

nition,� Proc. IEEE, vol. 77, no.2, Feb. 1989, pp.257-285.

[8] Z. Ghahramani et al., �Factorial hidden Markovmodels,� Machine Learning, vol. 29, no. 2-3, pp.245-273, Nov./Dec. 1997.

[9] F. M. Puglisi, P. Pavan, �RTN analysis withFHMM as a tool for multi-trap characterizationin HfOX RRAM,� Proc. IEEE Int. Conf. Elec-tron. Devices Solid-State Circuits. 2013 , pp. 1-2.

[10] G. Casella et al., �Explaining the Gibbs Sampler�American Statistician, vol. 46, no. 3, pp. 167-174, Aug. 1992.

Francesco M. Puglisi was born inCosenza, Italy, in 1987. He receivedthe B.S. and M.S. degrees in electronicengineering summa cum laude in 2008and 2010, respectively, from Universitàdella Calabria, Rende, Italy. He is cur-rently working toward the Ph.D. de-gree at Università di Modena e ReggioEmilia, Modena, Italy, in the Diparti-mento di Ingegneria �Enzo Ferrari� since2012. His work focuses on characteri-

zation of emerging non-volatile memories, especially RRAMs,and complex noise analysis, particularly RTN, and both com-pact and physics-based modeling. Mr. Puglisi is currentlyserving as a reviewer for IEEE Transactions on Device andMaterials Reliability. He was awarded with the �E. LoizzoMemorial Award� for being the best Master student of the en-gineering faculty at Università della Calabria during the 2010-2012 timespan. He was the recipient of the Best Student PaperAward at the IEEE ICICDT 2013 Conference.

Paolo Pavan was born in Italy in1964. He graduated in Electrical En-gineering at the University of Padova,Italy, in 1990 working on latch-up andhot-electron degradation phenomena inMOS devices. In 1991 he started hisPhD program studying impact ioniza-tion phenomena in advanced bipolartransistor and received his PhD in 1994from the same University. From 1992 to1994 he was at the University of Cali-

fornia at Berkeley where he studied radiation e�ects on MOSdevices and circuits. In 1998 he worked with Saifun Semi-conductors, in Israel, on the development of NROM, a newnonvolatile memory device. His research interests are in thecharacterization and modeling of Flash memory cells and onthe development of new nonvolatile cells and, more recently,in the development of safety critical and wireless applicationsfor automotive electronics. His activity is strongly connectedto companies and start-ups in the hi-tech business. He parte-cipates to many research projects, national and european. In2002-2003 he was in the IEDM Technical Committee �CMOSand Interconnect Reliability,� and Chair it in 2004. EuropeanCo-Chair of IEDM 2005 and 2006. He has been Chairmanof the Technical Committee �Nonvolatile and ProgrammableDevice Reliability� for ESREF 2002, and Guest Editor of theIEEE Transactions on Device and Material Reliability SpecialIssue on Nonvolatile Memories in Sept. 2004. He is in the Tech-nical Committee of VLSI-TSA from 2006 to 2010. He is in theTechnical Committe of ESREF 2012 and IRPS 2014; in ESS-DERC from 2012, and Technical Program Chair of ESSDERC2014. Prof. Pavan has been the President of the Italian Univer-sity Nano Electronics Team (IU.NET) Consortium, Bologna,Italy from 2005 to 2011. He authored and co-authored manytechnical and invited papers, one book and two chapters in

Factorial Hidden Markov Model analysis of Random Telegraph Noise in Resistive Random Access Memories 29

edited books; he gave seminars and short courses at interna-tional conferences and schools. He is currently Professor ofElectronics at the University of Modena and Reggio Emilia,Italy, where he acted as deputy dean for his College and De-partment and he has been member of the Academic Senate.He is currently Dean of the Electronics Engineering Program.