tailoring of random networks and graphs · 2017-09-25 · tailoring of random networks and graphs...

Tailoring of Random Networks and GraphsACC Coolen, King’s College London

Eindhoven, Sept 21st 2017

945

805

422

846 969

883

351

968

804

155

472

724263

19152

427

336 923

591 186

973124

574

32622198

920677

267

898

676

129

221 166

970647

814651

951

813

285

662

273

575

790

91

560

459

854

360

114

579

675

466

561

568

765

37

284

173

518

286

544

553

385

783 644

381870

626 278 89824

347 954

229

369

737

90

921

148

698

398

998103

586

390

531

222849

641149

101

255417

11

584

146418

39

880992391

260107

529

332

436

493845

623

25750

1000 14772

141

669

993 503

268

934314

41 214654

657

450

424

97

99

942

132

210

766725

387

2

781703

32 911

714

628

486429

837

328399

70583

792

337 339

236

983

59548810

400201

618

580

875245

825372

131335

599110

228

213

374

674

27

723

295

184

795509

745

630

631

371

728

625320

828

483

602

206665

751 540463

680

742

205

776

893

904

249

541

648498

346

482

473

402 75

294471

112771 169

848627

338

635

882

100609

378

925

643

364

403

353

528

987

931

6

590

747

154181

985642

569 519460

426

994

554456

866

587160

492563 545

47764

153

10

963522

506

401

811

489

650

578652

720

193167

543936

52

116

859

109

3246

892

270871

275

370

943

145

582

345

817

655

863572

573

380

940

595

794

559171

212

494

818

585307161

678

4 684

694 349

752

77511

237

982

822

978

458

151

40108

5

721660

407523

700121

607894

692

820327

156

949734

558

775441

490

850

262 45

929

36

331

313354

7

82

67 84690 119

157

137537

577

227

428767

477

123

855

88

66

757

788 51

216 344

961681 589

233889

999

856

28

128

224

991

225

204

226

756

17296910

6929

389

203

619 26

64

731

965

836

633862

876697

33231355

958

50722

18411

168

538

115

549

290

479

384

431624

253 113

603

802

215

533

81

668639

738 379

446

73

593

382

521

620164

423

939

373

608

140

325480

661

924

439

55760

656475

763

562

947

865

159 807

254

887

269

896

596

31

298682

565

54

966

964

505812178

177487

840

179

713

491905

570

796

604

15499 513

996

869

362

901

663

367

95

9

874

24826498

739990

279

950

932

534

550

406

867576809975 240

542

617

165

134

211 858302

241

826

469

547

808

843468

368

125

928497

93304

912

971495

916

821

357937

319

671

683

907 755

334

106

359

454

989191

917

967799

21864330

413

615769

375

478

13949467

777

797

741

748

762

415959

194

311687

956

247

616396

673

784

185

525

829

833

705

465

76

986

636501

170

420

918

63 744342438

984

250

852

614348

288

303

256

85293

176

666708

333

280

310

388

442

760356

20

879

861

323640

259512

957

743

835

71

80

197634

567162

358

709815

532

844

274

474461

502

689526

832

196

801

909995 276 42

571 200

257

282

535

455 12485

208

592318

632397

736

4815248692

778

444

816

847142 507

242

220

350

405

234

23961440

941

321

600

504

147

104

244283

834

853

277

653445 927

365 419803

530

182715

261704

707 35

395

696

895

555

913508

758

291

732

718

536

28916

305791

174

667786

980

944552

740 13

62

433

733727363

78287 839

780

299 800

126793

952464

44

322476

265

38

900

449190 392

601 915

135377

831

611

343

272

527

693

317

297

316

324

235

706759 827

138

412

120

594

496

68

443789

919962 878

649

300

111838 899

281

117

451

43 712

252

851

488622

890

386306

383

659

629

761432

312

144906

972 872735

691 470

309

437897

56

46

868

664

520

566

105271414

779

83

773

754

95548953514

729150

922888

979

806

701

188908207

376

914

308

546

98194

726

58787

613

646

143

597

782637

17

404340

717658

645

976

122

421

74

730

881 695

702292581

551

408484

556

189

163

716857

515884

53

770

199885

136974

830

55

133

410

686500

447

564

243

768

183

158195

127

217118

296452

621

409

638

841

341

238

79

598

102130

202

672

679

946

719

688

258

798

823

5391

588

606

329930

977

301180

988612

57926

605

510

891

187

430

425860

699

30

219

87

746

685

192

232

842

394

517711

886175

315

352

997

251361

462 670

785

935

266435

749 960948

933

51634873

453

610

903

366

223

IntroductionNetworks and graphsTailored random graph ensembles

Counting tailored random graphsEntropy and complexityNondirected graphsDirected graphs

Generating tailored random graphsCommon algorithms and their problemsMCMC processes for hard-constrained graphs

Degree-constrained MCMC graph dynamicsBookkeeping of movesMobility of nondirected graphsDirected graphs

Tailoring loopy graph ensemblesMotivationSpectrally constrained ensemblesSolvable toy model

Networks and graphs

nodes (vertices): i , j ∈ 1, . . . ,N

links (edges): cij ∈ 0, 1no self-links: cii = 0 for all i

graph: c = cij

@@@

@@@

cij =1

•i • j

nondirected: ∀(i , j) : cij = cji

directed: ∃(i , j) : cij 6= cji

if we model real-world systems by random graphswe want these graphs to be realistic ...

i.e. to have appropriate domain-specific statistical characteristics

Quantify topology of nondirected graphs

I degrees,degree sequence: ki (c) =

∑j cij , k(c) = (k1(c), . . . , kN(c))

degree distribution: p(k |c) =1N

N∑i=1

δk,ki (c)

I joint degree statisticsof connected nodes

@@@

@@@

cij =1•

ki = k?

•

kj = k ′?W (k , k ′|c) =

1N〈k〉

∑ij

cijδk,ki (c)δk′,kj (c)

normalisation :∑

k,k′≥0

W (k , k ′|c) = 1

assortativity/dissortativity : C = 〈kk ′〉W − 〈k〉W 〈k ′〉W

Quantify topology of directed graphslinks become arrows

I degrees,degree sequences: k in

i (c) =∑

j cij , kin(c) = (k in1 (c), . . . , k in

N (c))

kouti (c) =

∑j cji , kout(c) = (kout

1 (c), . . . , koutN (c))

degree distribution:

ki → ~ki = (k ini , k

outi ) p(~k |c) =

1N

∑i

δ~k,~ki (c)

I joint in-out degree statisticsof connected nodes

?@@@R

@@R

?@@@

@@R cij =1•

~ki = ~k?

•

~kj = ~k ′?W (~k , ~k ′|c) =1

N〈k〉∑

ij

cijδ~k,~ki (c)δ~k′,~kj (c)

note:

W (~k , ~k ′|c) 6= W (~k ′, ~k |c)

Graph classificationvia increasingly detailedfeature prescription

'

&

$

%

G = 0, 112 N(N−1)'

&

$

%

〈k〉 = ...'

&

$

%p(k) = . . .#

" !

W (k, k′) = . . .

Tailoringrandom graphs

maximum entropy random graph ensembles,p(c) with prescribed values for 〈k〉, p(k), W (k , k ′), . . .

– proxies for real networks in stat mech models– complexity: how many graphs have same features as c? counting– hypothesis testing: graphs as null models generation

N =1000: 212 N(N−1)≈10150,364 graphs

(universe has ∼1082 atoms ...)

Tailored random graph ensembles

(i) set G of allowed graphs,(ii) probability measure p(c) on G

I Tailoring via hard constraints

impose values for observables: Ωµ(c) = Ωµ for µ = 1 . . . p

p(c|Ω) =δΩ(c),ΩN (Ω)

, N (Ω) =∑

c

δΩ(c),Ω (# graphs in ensemble)

with Ω = (Ω1, . . . ,Ωp)note:

maximises Shannon entropy Son G[Ω] = c| Ω(c) = Ω S = − 1

N〈k〉∑

c

p(c) log p(c)

eN〈k〉S[Ω] = e−

∑c

δΩ(c),ΩN (Ω)

(log δΩ(c),Ω−logN (Ω)

)= N (Ω)

I Tailoring via soft constraints

impose averages for observables: Ωµ(c) = Ωµ for µ = 1 . . . pp(c): maximum entropy, subject to constraints

p(c|Ω) = Z−1(Ω) e∑µ ωµΩµ(c), Z (Ω) =

∑c

e∑µ ωµΩµ(c)

parameters ωµ:to be solved from ∀µ :

∑c

p(c|Ω)Ωµ(c) = Ωµ

now all graphs c can emerge,but those with Ω(c) ≈ Ω are most likely

effective # graphs N (Ω) defined via entropy:

N (Ω) = eN〈k〉S[Ω], S[Ω] = − 1N〈k〉

∑c∈G

p(c|Ω) log p(c|Ω)

Example

nondirected graphs, cii = 0 for all i ,impose average connectivity via hard constraint,Ω(c) =

∑ij cij

I demand∑

ij cij = N〈k〉

p(c|〈k〉) =δ∑

ij cij ,N〈k〉

N (〈k〉) , N (〈k〉) =∑

c

δ∑ij cij ,N〈k〉

I calculate N (〈k〉):

use δnm = (2π)−1 ∫ π−πdω ei(n−m)ω

N (〈k〉) =

∫ π

−π

dω

2πeiωN〈k〉

∑c

e−iω∑

ij cij =( 1

2 N(N−1)12 N〈k〉

)= e

12 N〈k〉

[log(N/〈k〉)+1

]+O(log N)

Example

nondirected graphs, cii = 0 for all i ,impose average connectivity via soft constraint,Ω(c) =

∑ij cij

I demand 〈∑

ij cij〉 = N〈k〉

p(c|〈k〉) =1

Z (ω)eω

∑ij cij , Z (ω) =

∑c

eω∑

ij cij

ω solved from : 〈k〉 =d

dω

1N

log Z (ω)

I calculate Z (ω) and ω:

〈k〉 = (N−1)e2ω

e2ω+1

I rewrite probabilities:

p(c|〈k〉) =∏i<j

[ e2ω

e2ω+1δcij ,1 +

1e2ω+1

δcij ,0

]Erdos-Renyi ensemble

Counting tailored random graphs

'

&

$

%

G = 0, 112 N(N−1)'

&

$

%

〈k〉 = ...'

&

$

%p(k) = . . .#

" !

W (k, k′) = . . .

-PPPPPPPPPq

ZZZZZZZZ~

@@@@@@@@R

how many graphsin each family?

Note:

solving models of interacting particle systems on tailored random graphs(via replica method or generating functional analysis, for N→∞):feasible if we can compute the entropy of the graph ensemble!

entropy and complexity

I effective nr of graphs in ensemble p(c|Ω):(Ω: values of imposed observables)

N (Ω) = eN〈k〉S(Ω), S(Ω) = − 1N〈k〉

∑c∈G

p(c|Ω) log p(c|Ω)

I S(Ω): proportional to average nr of bits we need to specifyto identify a graph c in the ensemble

I complexity of graphs in ensemble p(c|Ω):

∃ many graphs with feature Ω: graphs with Ω have low complexity∃ few graphs with feature Ω: graphs with Ω have high complexity

C(Ω) = S(∅)− S(Ω)

∅: no constraintsnondirected, cii = 0 ∀i :

p(c|∅) = 2−12 N(N−1), S(∅) = − 1

N〈k〉 log 2−12 N(N−1) =

N−12〈k〉 log 2

Nondirected graphs

p(c) =∑

k

[∏i

dki p(ki )]∏

i δki ,ki (c)

Z (k,W )

∏i<j

[ 〈k〉N

W (ki , kj )

p(ki )p(kj )δcij ,1 +

(1− 〈k〉

NW (ki , kj )

p(ki )p(kj )

)δcij ,0

]

S =12

[1+log(N〈k〉 )]︸︷︷︸

Erdos−Renyi entropy

− 1〈k〉

∑k

p(k) log[p(k)

p(k)]︸︷︷︸

degree complexity

+12

∑k,k′

W (k , k ′) log[ W (k , k ′)

W (k)W (k ′)

]︸︷︷︸

wiring complexity

+ εN

limN→∞ εN = 0

p(`) = e−〈k〉〈k〉`/`!degree distr of Erdos-Renyi graphs

(path integrals, integral representations,steepest descent, ...)

Directed graphs

~ki = (k ini , k

outi )

p(c) =∑~k

∏i

[d~ki p(~ki )

]∏i δ~ki ,

~ki (c)

Z (~k,W )

∏i<j

[ 〈k〉N

W (~ki , ~kj )

p(~ki )p(~kj )δcij ,1 +

(1− 〈k〉

NW (~ki , ~kj )

p(~ki )p(~kj )

)δcij ,0

]

S = 1+log(N〈k〉 )︸︷︷︸

directed ER entropy

− 1〈k〉

∑~k

p(~k) log[p(~k)

p(k in)p(kout)]

︸︷︷︸degree complexity

+∑~k,~k′

W (~k , ~k ′) log[ W (~k , ~k ′)

W (~k)W (~k ′)

]︸︷︷︸

wiring complexity

+ εN

limN→∞ εN = 0

p(`) = e−〈k〉〈k〉`/`!p(k in)p(kout): degree distr of directed Erdos-Renyi graphs

(path integrals, integral representations,steepest descent, ...)

Generating tailored random graphs'

&

$

%

G = 0, 112 N(N−1)'

&

$

%

〈k〉 = ...'&

$%

k = . . .

W (k, k′) = . . .

G: all nondirected N-node graphsG[k]⊂G: all nondirected N-node graphs with degrees k

typical questions:how to generate numerically

I random c∈G, with specified p(c)

I random c∈G[k], with uniform p(c)

I random c∈G[k], with specified p(c)

similar for directed graphs ...

Common algorithms and their problems

soft constraints only:standard Glauber/Gibbs/MCMC dynamics

objective: generate random nondirected c ∈ 0, 112 N(N−1)

with specified probabilities p(c)

strategy: start from any graph cpropose random moves cij → 1−cij (giving c→ Fijc),

define acceptance probabilities A(Fijc|c)via detailed balance condition

A(Fijc|c)p(c) = A(c|Fijc)p(Fijc) → A(c′|c) =[1 + p(c)/p(c′)

]−1

stochastic process is ergodic, and converges to p(c)

practicalities:equilibration can take a very long time,so monitor Hamming distances similar for directed graphs ...

The problem of phase transitions

example:p(c) =

1Z (α, β)

eα∑

i ki (c)+β∑

i k2i (c), N =300, α=4

1

10

100

1000

100 1000 10000 100000 1e+06 1e+07

1

10

100

1000

100 1000 10000 100000 1e+06 1e+07

1

10

100

1000

100 1000 10000 100000 1e+06 1e+07

t t t

〈k〉

β=0.01 β=0.02 β=0.03

I phase transitions sometimes prevent us fromcontrolling observables in soft-constrained ensembles

I need hard constrained ensembles ...but these are harder to sample via MCMC ...

Matching algorithm(Bender and Canfield, 1978)

objective: generate random nondirected graph c ∈ 0, 112 N(N−1)

with specified degree sequence k = (k1, . . . , kN)

strategy: stochastic growth dynamics,starting from graph with no links

I initialisation: cij = 0 for all (i , j)

repeat:I pick at random two nodes (i , j)I if

∑` ci` < ki and

∑` cj` < kj :

connect i and jcij = 0 → cij = 1

terminate if∑

j cij = ki for all i ••

•

•

•

••

•

•

•

•

•

AA

AA

HH

JJHH JJ

HH

AA

?

(trivially generalisedto directed graphs)

Matching algorithmlimitations and problems ...

I major limitation:

aims to generate random c∈G[k],but cannot control graph probabilities ...

••

•

•

•

••

•

•

•

•

•

AA

AA

HH

JJHH JJ

HH

AA

?

I inconvenience: convergence not guaranteed

process can ‘hang’ before∑

j cij = ki for all iif a remaining ‘stub’ requires self-loops– monitor evolving degrees, to test for this– if process ‘hangs’: reject and start again from empty graph

I sampling bias:

if process ‘hangs’, users often don’t reject the graphbut do ‘backtracking’ (for CPU reasons),this creates correlations between graph realisations

even if we reject rather than backtrack:no proof published yet that sampling measure p(c) is flat ...

MCMC with hard constraints

need to think more carefully aboutelementary moves in space of graphs

MOVE SET INVARIANTS ACTION

Link flipsFij

none••i

j↔ ••i

j

Hinge flipsFijk

average degree

k(c) = 1N

∑rs crs

•••j

i

kAA ↔ •

••j

i

k

Edge swapsFijk`

all individual degreeski (c) =

∑j cij , i = 1 . . .N

• •• •

j k

i `↔ ••••

j k

i `

Edge switching algorithm(Seidel, 1976)

objective: generate random nondirected graph c ∈ 0, 112 N(N−1)

with specified degree sequence k = (k1, . . . , kN)

strategy: degree-preserving randomisation (‘shuffling’) process,starting from any graph k = (k1, . . . , kN)

I initialisation: cij = c0ij for all (i , j),

c0: any graphwith the correct degrees

repeat:I pick at random four nodes (i , j , k , `)

that are pairwise connectedI carry out an ‘edge swap’

(or ‘Seidel switch), see diagram(preserves all degrees!)

terminate if stochastic process has equilibrated

• •

• •−→

• •

• •

i

j

k

`

i

j

k

`

Edge switching algorithmlimitations and problems ...

I major limitation:

aims to generate random c∈G[k],but cannot control graph probabilities ...

• •

• •−→

• •

• •

i

j

k

`

i

j

k

`

I inconvenience: need for a ‘seed graph’with the correct degrees k = (k1, . . . , kN)

I sampling bias:

edge swaps are ergodic on G[k] (Taylor, 1981),but sampling is not uniform!

many possible moves few moves ...

nr of possible movesdepends on state c!

result:stationary state of Markov chainfavours high-mobility graphs

dangerous for scale-free graphs ...

target:uniform measure p(c)on G[k]

n(c) : nr of possible moves

n(c) = (N−2)(N−3)

1 graphn(c) = 2(N−3)

(N−2)(N−3) graphs

0.01

0.02

0.03

0 100000 200000 300000 400000 500000

←simulation

←theoryn(c)/N2

executed moves

for flat measure:

n(c) =(N−2)(N−3)[1 + 2(N−3)]

1 + (N−2)(N−3)

N = 100:n(c)/N2 ≈ 0.0195

‘accept all’edge swapping:

why is the generation of graphswith hard constraints nontrivial?

I many users underestimate/misjudge what the real problem is:

sampling space of all graphs with given features: usually easy ...sampling them with specified probabilities: nontrivial!

I many ad-hoc graph generation algorithms appear sensible,but lack analysis of which measure they converge to

random graphs are often used as ‘null models’,against which to test hypotheses on real networks

if null model is biased,hypothesis test is fundamentally flawed ...

MCMC processes for hard-constrained graphs

I hard constraints:G[?] ⊆ G: all c∈G that satisfy constraints ?

I stochastic graph dynamics as a Markov chain,transition probabilities W (c|c′) for move c′ → c

∀c ∈ G[?] : pt+1(c) =∑

c′∈G[?]

W (c|c′)pt (c′)

I allowed moves (exclude identity):

Φ: set of allowed moves F : GF [?]→ G[?]GF [?]: those c ∈ G[?] on which F can act

all moves are auto-invertible: (∀F ∈ Φ) : F 2 = 1IΦ is ergodic on G[?]

objectiveconstruct transition probs on G[?], based on move set Φ,such that process converges to p(c) = Z−1e−H(c)

canonical Markov chain

ergodic auto-invertible moves F ∈ Φ,convergence to p(c) = Z−1e−H(c) on G[?]for acceptance probabilities

A(c|c′) =n(c′)e−

12 [H(c)−H(c′)]

n(c′)e−12 [H(c)−H(c′)] + n(c)e

12 [H(c)−H(c′)]

naive edge-swapping?(∀c, c′) : A(c|c′) = 1

(∀F , c) :A(Fc|c)e−H(c)

n(c)=

A(c|Fc)e−H(Fc)

n(Fc)→ (∀F , c) :

e−H(c)

n(c)=

e−H(Fc)

n(Fc)

corresponds toH(c)=− log n(c),so would give sampling bias : p(c) =

n(c)∑c′∈G[?] n(c′)

picking moves randomly ...

correct sampling: q(F |c) = 1/n(c)for all possible moves

PROTOCOL 1:(i) pick a site j with kj (A) > 0(ii) pick a site i ∈ ∂j (A)(iii) pick a site k /∈ ∂i (A)∪i

•j → ••

j

iAA → •

••j

i

kAA

PROTOCOL 2:(i) pick two disconnected sites

(i , k) with ki (A) > 0(ii) pick a site j ∈ ∂i (A)

••

i

k→ •••j

i

kAA

PROTOCOL 3:(i) pick two connected sites (i , j)

and a third site k(ii) while Aik = 1 return to (i)

•••j

i

kAA

0 10 20 30 40 50 60 700

0.05

0.1

0.15

0.2

InitialFinalTarget

0 10 20 30 40 50 60 700

0.05

0.1

0.15

0.2

InitialFinalTarget

0 10 20 30 40 50 60 700

0.05

0.1

0.15

0.2

InitialFinalTarget

k

p(k)

p(k)

p(k)

PROTOCOL 3

PROTOCOL 2

PROTOCOL 1

N =3000, 〈k〉=7

dashed: start graphdotted: p(k) of target p(A)

solid: MCMC result

Bookkeeping of moves

I constraints: imposed degrees k

ergodic set Φ of admissible moves:edge swaps F : GF [k]→ G[k]

(i , j , k , `) ∈ 1, . . . ,N4| i< j<k<`, ordered node quadruplets

I

s ss si j

k`

II

s ss si j

k`@@

III

s ss si j

k`

@@@@@@

IV

s ss si j

k`

V

s ss si j

k`@@@@@@

VI

s ss si j

k`

@@

I group into pairs (I,IV), (II,V), and (III,VI)auto-invertible swaps: Fijk`;α, with i< j<k<` and α∈1, 2, 3

Iijk`;α(c) = 1:Fijk`;α(c)qr = 1−cqr for (q, r) ∈ Sijk`;α

Fijk`;α(c)qr = cqr for (q, r) /∈ Sijk`;α

Sijk`;1 =(i , j), (k , `), (i , `), (j , k), Sijk`;2 =(i , j), (k , `), (i , k), (j , `)Sijk`;3 =(i , k), (j , `), (i , `), (j , k)

Mobility of nondirected graphsto implement the Markov chain,need analytical formula for the graph mobility

n(c) =∑N

i<j<k<`

∑3α=1 Iijk`;α(c)

Iijk`;1(c) = cijck`(1−ci`)(1−cjk ) + (1−cij )(1−ck`)ci`cjk

Iijk`;2(c) = cijck`(1−cik )(1−cj`) + (1−cij )(1−ck`)cik cj`

Iijk`;3(c) = cik cj`(1−ci`)(1−cjk ) + (1−cik )(1−cj`)ci`cjk

work out combinatorics:

n(c) =14

N2〈k〉2 +14

N〈k〉 − 12

N〈k2〉︸︷︷︸invariant

+14Tr(c4) +

12Tr(c3)− 1

2

∑ij

kicijkj︸︷︷︸state dependent

I state-dependent part can be ignored if 〈k2〉kmax/〈k〉2 NI avoid calculating n(c) at each iteration step:

(i) calculate n(c) at time t = 0(ii) update dynamically, compute ∆ijk`;αn(c) for executed move Fijk`;α

Example:target =uniform measure on G[k]

many possible moves few moves

0.01

0.02

0.03

0 100000 200000 300000 400000 500000

A(c|c′) = 1

A(c|c′) = [1+ n(c)n(c′) ]−1

n(c)/N2

executed moves

N = 100

naive versus correctacceptance probabilities

predictions:

p(c) = constant :

n(c)/N2≈ 0.0195

p(c) = n(c)/Z :

n(c)/N2≈ 0.0242

Exampletarget =degree-correlatedmeasure on G[k]

0.00

0.05

0.10

0.15

0.20

0 10 20 30 40

P(k)

k

0.00.10.20.30.40.50.60.70.80.91.01.11.21.31.41.51.61.71.81.92.0

1 5 10 15 20 25 30

1

5

10

15

20

25

3030

k′

k

Π(k , k ′|c0)

0.00.10.20.30.40.50.60.70.80.91.01.11.21.31.41.51.61.71.81.92.0

1 5 10 15 20 25 30

1

5

10

15

20

25

3030

k′

k

Π(k , k ′) (target)

0.00.10.20.30.40.50.60.70.80.91.01.11.21.31.41.51.61.71.81.92.0

1 5 10 15 20 25 30

1

5

10

15

20

25

3030

k′

k

Π(k , k ′|cfinal)

N = 4000,〈k〉 = 5

Π(k , k ′) =(k − k ′)2

[β1 − β2k + β3k2][β1 − β2k ′ + β3k ′2]

Directed graphs

bookkeeping of elementary movesI constraints: imposed in-out degrees, so graph set is G[kin, kout]

set Φ of admissible moves:directed edge swaps F : GF [kin, kout]→ G[kin, kout]

u uu uix jx

jyiy

-

-⇔ u uu uix jx

jyiy@@@R

for nondirected graphs:edge swaps are ergodic set of moves(Taylor, 1981 – proof based on Lyapunov function)

Rao, 1996:

unless self-interactions are allowed,edge swaps not ergodic for directed graphs

further move type requiredto restore ergodicity:3-loop reversal u u

uix jx = iy

jy

-@@@I?

⇔ u uu

ix jx = iy

jy6

@@@R

to implement the Markov chain,need to calculate graph mobility analytically:

n(c) =

12

N2〈k〉2 −∑

j

k inj kout

j︸︷︷︸invariant

+12Tr(c2) +

12Tr(c†cc†c) + Tr(c2c†)−

∑ij

k ini cijkout

j︸︷︷︸state dependent

n4(c) =13Tr(c3)− Tr(cc2) + Tr(c2c)− 1

3Tr(c3)︸︷︷︸

state dependentwith: (c†)ij = cji , cij = cijcji

Motivationis our ‘tailoring’ adequate?

e.g. do we recover phase transitions ofIsing models on tailored random graphs?

ΩA: correct 〈k〉ΩB : correct p(k)

ΩC : correct p(k) and W (k , k ′)

I c? = d-dim cubic latticep(k) = δk,2d

0

2

4

6

8

10

ΩA ΩB ΩC

Tc(d)

Tc (1) = 0

Tc (2) = 2/log(1+√

2)

Tc (3)≈4.512

Tc (4)≈6.687

I c? = ‘small world’ latticep(k≥2) = e−qqk−2/(k−2)!

0

1

2

3

4

5

6

ΩA ΩB ΩC

Tc(q)

Tc (0) = 0

Tc (1) = 2/log(1+√

2)

Tc (2) = 2/log( 34 + 1

4√

17)

Tc (3) = 2/log( 23 + 1

3√

7)

It is all about short loops ...

critical temperatures Tc(d)

degrees 4-loops d =1 d =2 d =3 d =4

random, 〈k〉=2d 1.820 3.915 5.944 7.958random, p(k)=δk,2d X 0 2.885 4.933 6.952hypercubic Bethe X X 0 2.771 4.839 6.879

true cubic lattice X X 0 2.269 4.511 6.680

hypercubic Bethe lattice:‘tree of hypercubes’

– correct local degrees– geometric (non-random)– finite nr of short loops per site

I maximum entropy random graphs withprescribed p(k), W (k , k ′): locally tree-like ...

I more realistic graph tailoring:constrain nr of short loops

problem: most analysis methods, e.g.replicas, GFA, cavity method, belief prop, etcrequire locally tree-like graphs(modulo loop corrections)

exceptions:cubic lattices d<3

spherical modelsrecent immune models

Spectrally constrained ensemblesI control closed paths

of all lengths p(c) =1Zδk,k(c) e

∑`≥3 α`

∑i1...i`

ci1 i2ci2 i3

...ci` i1

generating function:

φ =1N

log∑

c

δk,k(c) e∑`≥3 α`Tr(c`)

〈m`〉 = 1N

⟨Tr(c`)

⟩= ∂φ/∂α`

S = φ−∑`≥3 v`〈m`〉

I Tr(c`) = N∫dµ µ`%(µ|c),

so we control spectrum %(µ):p(c) =

1Zδk,k(c) e

N∫

dµ %(µ)%(µ|c)

%(µ) =∑`≥3

α`µ`

generating function:

φ =1N

log∑

c

δk,k(c) eN

∫dµ %(µ)%(µ|c)

%(µ) = δφ/δ%(µ)

S = φ−∫dµ %(µ)%(µ)

Some relevant questions'

&

$

%

〈k〉= . . .'

&

$

%

(k1, . . . , kN)= . . .

%(µ)= . . .

I Q1: How informative are spectra of finitely connected graphs?

I Q2: How many non-isomorphic graphs are there withgiven degrees (k1, . . . , kN) and a given spectrum %(µ)?

I Q3: How similar are processes running on non-isomorphic graphs withthe same degrees (k1, . . . , kN) and the same spectrum %(µ)?

(spherical spins: free energies identical!)

how to compute

φ =1N

log∑

c

δk,k(c) eN∫dµ %(µ)%(µ|c)

Analyticalroute forward p(c) =

1Z

δk,k(c) eN∫dµ %(µ)%(µ|c)

I Edwards-Jones:

%(µ|c) =2

Nπlimε↓0

Im∂

∂µlog Z (µ+iε|c), Z (µ|c) =

∫dφ e−

12 iφ·[c−µ1I]φ

I insert, integrate by parts,discretize µ-integral:

φ =1N

log∑

c

δk,k(c) eN∫dµ %(µ) 2

Nπ limε↓0 Im∂∂µ log Z (µ+iε|c)

= limε,∆↓0

1N

log∑

c

δk,k(c)

∏µ

e−2Im log Z (µ+iε|c). ∆π

ddµ %(µ)

I e−2 Im log z = z i.z −i

φ = limε,∆↓0

1N

log∑

c

δk,k(c)

∏µ

[Z (µ+iε|c)i Z (µ+iε|c)

−i]∆π

ddµ%(µ)

φ = limε,∆↓0

1N

log∑

c

δk,k(c)

∏µ

[Z (µ+iε|c)n(µ) Z (µ+iε|c)

m(µ)],

n(µ) = i∆π

ddµ%(µ)

m(µ) = − i∆π

ddµ%(µ)

I replica method:factorization over entries cij(products of Gaussian integrals)

I steepest descent for N→∞,continuation to imaginary dimensions,limits ε↓0 and ∆↓0

I replica symmetry, bifurcation analysis,phase transitions and entropy

I elegant order parameter equations,interpretation in terms of ‘loopy’ message passing with a twist,treelike results (entropy, spectrum, ...) all recovered for %(µ)→ 0

but still wrong ... !?

Solvable toy model

simplest member of the familiy:

k = (2, . . . , 2), %(µ) =K∑`=3

α`µ` : p(c) =

e∑K`=1 α`Tr(c`)

Z (α)

N∏i=1

δ∑j cij ,2

control nr of closed paths up to length K in 2-regular graphs ...

I all 2-regular graphs c: collections of rings,combinatorics solvable:

limN→∞

φN = limN→∞

extrω[iω +

K∑`=3

e(α`−iω)`

2`N+

N∑`=K +1

e−iω`

2`N

]densities m` of length ` ≤ K closed pathsalways vanish for N →∞ ...

I Canonical parameter scaling: α` = α` + `−1 log(N)

ϕ(α) = limN→∞

extrωiω +

K∑`=3

e(α`−iω)`

2`+

N∑`=K +1

e−iω`

2`N

p(c) =e∑K`=1

(`−1 log N+α`

)Tr(c`)

Z (α)

N∏i=1

δ∑j cij ,2

I finite densities m` of closed paths

I two phases, critical manifold: 1 = 12

∑K`=3 e

`α`

disconnected (large α) : no extensively large rings, only small loops

connected (small α) : extensively large rings exist

−0.2 0.0 0.2 0.4 0.6 0.8

0.2

0.4

0.6

0.8

1.0

m3

α3

K = 3

simulations: N = 1000, 5000

solid line: theory

lesson for spectrally constrained ensembles:

p(c) =1Zδk,k(c) e

N∫

dµ %(µ)%(µ|c), %(µ)→ %(µ) log N + %(µ)

Some references

Entropies of tailored graph ensemblesBianconi, Coolen, Perez-Vicente, Phys Rev E 78, 2008Annibale, Coolen, Fernandes, Fraternali, Kleinjung, J Phys A 42, 2009Roberts, Schlitt, Coolen, J Phys A 44, 2011Roberts, Coolen, J Phys A 47, 2014

Graph dynamics and generationTaylor, Combinatorial Mathematics VIII, Springer Lect Notes Math 884, 1981Rao, Jana, Bandyopadhya, Indian J of Statistics 58, 1996Coolen, De Martino, Annibale, J Stat Phys 136, 2009Roberts, Coolen, Phys Rev E 85, 2012

Coolen, Annibale, Roberts,Generating random networks and graphs(Oxford University Press, 2017)

Loopy graph ensemblesCoolen, J Phys Conference Series 699, 2016Aguirre-Lopez, Barucca, Fekom, Coolen,

arXiv:1705.03743, 2017 https://nms.kcl.ac.uk/ton.coolen

tailoring of random networks and graphs · 2017-09-25 · tailoring of random networks and graphs...

Documents