Skip to content

Commit 496187a

Browse files
andypetrellaandypetrella
committed
Merge branch 'master' into deanwampler-master
2 parents e060fe1 + 08939d5 commit 496187a

File tree

1 file changed

+125
-33
lines changed

1 file changed

+125
-33
lines changed

‎notebooks/Why Spark Notebook.snb‎

Lines changed: 125 additions & 33 deletions
Original file line numberDiff line numberDiff line change
@@ -35,19 +35,35 @@
3535
"id" : "9D4B9FB7CE8547AF8E22DEFB2F004560"
3636
},
3737
"cell_type" : "markdown",
38-
"source" : "Before getting started, the Spark Notebook is also famous for \n* its great community out of its 1300+ stars on GitHub,\n* its very active [gitter](https://gitter.im/andypetrella/spark-notebook) channel (1st June 2016): \n * 460+ participants and \n * 750+ messages per month."
38+
"source" : "Before getting started, the Spark Notebook is also famous for \n* its great community out of its 1300+ stars on GitHub,\n* its very active [gitter](https://gitter.im/andypetrella/spark-notebook) channel: \n * 460+ participants and \n * 750+ messages per month.\n\n<small>(1st June 2016)</small>"
3939
}, {
4040
"metadata" : {
4141
"id" : "816B3C0AC858439AB425EE1BBCF3F553"
4242
},
4343
"cell_type" : "markdown",
4444
"source" : "<a href=\"https://gitter.im/andypetrella/spark-notebook?utm_source=badge&amp;utm_medium=badge&amp;utm_campaign=pr-badge&amp;utm_content=badge\"><img src=\"https://camo.githubusercontent.com/da2edb525cde1455a622c58c0effc3a90b9a181c/68747470733a2f2f6261646765732e6769747465722e696d2f4a6f696e253230436861742e737667\" alt=\"Gitter\" data-canonical-src=\"https://badges.gitter.im/Join%20Chat.svg\" style=\"max-width:100%;\"></a>"
45+
}, {
46+
"metadata" : {
47+
"id" : "4FB943930C654421BF949217B72BC5CE"
48+
},
49+
"cell_type" : "markdown",
50+
"source" : "---\n## Skin to control the flow visually"
51+
}, {
52+
"metadata" : {
53+
"trusted" : true,
54+
"input_collapsed" : false,
55+
"collapsed" : false,
56+
"id" : "E11445B1CCCE465B83A27E374CCF5BED"
57+
},
58+
"cell_type" : "code",
59+
"source" : ":html\n<style>h2{color: coral; } hr{height: 3px; background-color: cornflowerblue !important; }</style>",
60+
"outputs" : [ ]
4561
}, {
4662
"metadata" : {
4763
"id" : "ADA0BB6570A04F9A832BC2A4A1827441"
4864
},
4965
"cell_type" : "markdown",
50-
"source" : "## Multiple Spark Contexts"
66+
"source" : "---\n## Multiple Spark Contexts"
5167
}, {
5268
"metadata" : {
5369
"id" : "24F6BEA80211494580BE0C17D62C995E"
@@ -66,7 +82,7 @@
6682
"input_collapsed" : false,
6783
"collapsed" : true,
6884
"presentation" : {
69-
"tabs_state" : "{\n \"tab_id\": \"#tab1345575921-0\"\n}",
85+
"tabs_state" : "{\n \"tab_id\": \"#tab624392215-0\"\n}",
7086
"pivot_chart_state" : "{\n \"hiddenAttributes\": [],\n \"menuLimit\": 200,\n \"cols\": [],\n \"rows\": [],\n \"vals\": [],\n \"exclusions\": {},\n \"inclusions\": {},\n \"unusedAttrsVertical\": 85,\n \"autoSortUnusedAttrs\": false,\n \"inclusionsInfo\": {},\n \"aggregatorName\": \"Count\",\n \"rendererName\": \"Table\"\n}"
7187
},
7288
"id" : "8BCC90AE74CF4C00811E64FD82287703"
@@ -99,7 +115,7 @@
99115
"id" : "6D96AEA2A64E48E383FCEF609B635966"
100116
},
101117
"cell_type" : "markdown",
102-
"source" : "## Metadata"
118+
"source" : "---\n## Metadata"
103119
}, {
104120
"metadata" : {
105121
"id" : "1B7ACC1DDB8348688F7FF4D7895727B3"
@@ -225,7 +241,7 @@
225241
"id" : "7F3F63814584413E817A6AE5B56C0FAD"
226242
},
227243
"cell_type" : "markdown",
228-
"source" : "## Logs"
244+
"source" : "---\n## Logs"
229245
}, {
230246
"metadata" : {
231247
"id" : "D4B5803CA95D4E6B9497CA86DE6C866F"
@@ -237,19 +253,29 @@
237253
"id" : "A187C1B705A644B794DDEDB33AE7B223"
238254
},
239255
"cell_type" : "markdown",
240-
"source" : "## Side pane"
256+
"source" : "---\n## Side pane"
241257
}, {
242258
"metadata" : {
243259
"id" : "9870E3C823B048CC8D558F2B538CC83F"
244260
},
245261
"cell_type" : "markdown",
246-
"source" : "In the **View** menu, you'll can open the side pane which contains many interesting panels:\n* terms: a table listing the defined _functions_, _variables_ and _types_ !\n* error logs: displaying and bringing back any errors thrown in the server\n* chat room: a fancy chat room available for the current notebook (see below in the synchronized section)"
262+
"source" : "In the **View** menu, you'll can open the side pane which contains many interesting panels:\n* `terms` : a table listing the defined _functions_, _variables_ and _types_ !\n* `error logs` : displaying and bringing back any errors thrown in the server\n* `chat room` : a fancy chat room available for the current notebook (see below in the synchronized section)"
263+
}, {
264+
"metadata" : {
265+
"trusted" : true,
266+
"input_collapsed" : false,
267+
"collapsed" : true,
268+
"id" : "90AF0CFA143E4F6283502273EF5DB351"
269+
},
270+
"cell_type" : "code",
271+
"source" : ":javascript\njQuery('a#toggle-sidebar').click()",
272+
"outputs" : [ ]
247273
}, {
248274
"metadata" : {
249275
"id" : "3F098E2A54C446E486EE544C75C756F9"
250276
},
251277
"cell_type" : "markdown",
252-
"source" : "## Plotting"
278+
"source" : "---\n## Plotting"
253279
}, {
254280
"metadata" : {
255281
"id" : "39F23F6B83C6465D8DE2CD5BD18C2C41"
@@ -270,7 +296,7 @@
270296
"id" : "3B03DFAD401D44B08D0C8592C5246AF0"
271297
},
272298
"cell_type" : "code",
273-
"source" : "case class Example(id:Int, category:String, value:Long, advanced:Boolean)\nimport scala.util.Random\nimport scala.util.Random._\nval categories = List.fill(5)(List.fill(10)(nextPrintableChar).mkString)\ndef category:String = shuffle(categories).head\nval examples = List.fill(100)(Example(nextInt(2000), category, nextLong, nextBoolean))",
299+
"source" : "case class Example(id:Int, category:String, value:Long, advanced:Boolean)\nimport scala.util.Random._\n\nval categories = List.fill(5)(List.fill(10)(nextPrintableChar).mkString)\ndef category:String = shuffle(categories).head\nval examples = List.fill(100)(Example(nextInt(2000), category, nextLong, nextBoolean))",
274300
"outputs" : [ ]
275301
}, {
276302
"metadata" : {
@@ -284,7 +310,7 @@
284310
"input_collapsed" : false,
285311
"collapsed" : true,
286312
"presentation" : {
287-
"tabs_state" : "{\n \"tab_id\": \"#tab955364267-0\"\n}",
313+
"tabs_state" : "{\n \"tab_id\": \"#tab1819083238-0\"\n}",
288314
"pivot_chart_state" : "{\n \"hiddenAttributes\": [],\n \"menuLimit\": 200,\n \"cols\": [],\n \"rows\": [],\n \"vals\": [],\n \"exclusions\": {},\n \"inclusions\": {},\n \"unusedAttrsVertical\": 85,\n \"autoSortUnusedAttrs\": false,\n \"inclusionsInfo\": {},\n \"aggregatorName\": \"Count\",\n \"rendererName\": \"Table\"\n}"
289315
},
290316
"id" : "2E61401D7F6B4DCEA0D5F52DE96FB973"
@@ -315,7 +341,7 @@
315341
"id" : "595E9615FC9A410AA16F64B33FF80912"
316342
},
317343
"cell_type" : "code",
318-
"source" : "PivotChart(examples)",
344+
"source" : "TableChart(examples)",
319345
"outputs" : [ ]
320346
}, {
321347
"metadata" : {
@@ -364,7 +390,7 @@
364390
"id" : "4153FEFD93464E86A0F7FF113FB1139F"
365391
},
366392
"cell_type" : "markdown",
367-
"source" : "## Graphs"
393+
"source" : "---\n## Graphs"
368394
}, {
369395
"metadata" : {
370396
"id" : "F91D80C4BFD0408DA915051BC3AEE740"
@@ -385,6 +411,16 @@
385411
"cell_type" : "code",
386412
"source" : "val catCol = categories.zip(List(\"#000\", \"#478\", \"#127\", \"#984\", \"#F5A\")).toMap",
387413
"outputs" : [ ]
414+
}, {
415+
"metadata" : {
416+
"trusted" : true,
417+
"input_collapsed" : false,
418+
"collapsed" : true,
419+
"id" : "E31A5ACA4F8D442383289F17C620C5B4"
420+
},
421+
"cell_type" : "code",
422+
"source" : "val nodes = examples.map(e => Node(e.id, e, catCol(e.category), 5))\n\n\nval clusters = examples.groupBy(_.category)\nval connectedClusters = clusters.toList.flatMap { case (c, cl) => \n for {\n a <- cl\n b <- cl if a != b\n } yield Edge(2000+a.id+b.id, (a.id, b.id), \"yellow\")\n }\n\n\nval singleConnections = {\n val s = clusters.toList.map(_._2).map(x => shuffle(x).head)\n \n for (a <- s; b <- s if a != b) \n yield Edge(4000+a.id+b.id, (a.id, b.id), \"green\")\n}\n\nval all = nodes ::: connectedClusters ::: singleConnections",
423+
"outputs" : [ ]
388424
}, {
389425
"metadata" : {
390426
"trusted" : true,
@@ -397,14 +433,14 @@
397433
"id" : "1F073EEB99CE4B1781AD74C59E208805"
398434
},
399435
"cell_type" : "code",
400-
"source" : "val nodes = examples.map(e => Node(e.id, e, catCol(e.category), 5))\nval clusters = examples.groupBy(_.category)\nval connectedClusters = clusters.toList.flatMap { case (c, cl) => \n for {\n a <- cl\n b <- cl if a != b\n } yield Edge(2000+a.id+b.id, (a.id, b.id), \"yellow\")\n }\nval singleConnections = {\n val s = clusters.toList.map(_._2).map(x => shuffle(x).head)\n for (a <- s; b <- s if a != b) yield Edge(4000+a.id+b.id, (a.id, b.id), \"green\")\n}\n\nval all = nodes ::: connectedClusters ::: singleConnections\nGraphChart(all, maxPoints = all.size)",
436+
"source" : "GraphChart(all, maxPoints = all.size)",
401437
"outputs" : [ ]
402438
}, {
403439
"metadata" : {
404440
"id" : "E14D74FA3B164544A79D22800BB84BDD"
405441
},
406442
"cell_type" : "markdown",
407-
"source" : "## Geo charts"
443+
"source" : "---\n## Geo charts"
408444
}, {
409445
"metadata" : {
410446
"id" : "E78EF394884248DF87A887AFE627B13F"
@@ -467,12 +503,78 @@
467503
"cell_type" : "code",
468504
"source" : "val airportsDFWithStyles = airportsDF.withColumn(\"r\", when($\"state\" === \"AK\", 10).when($\"state\" === \"TX\", 9).when($\"state\" === \"CA\", 8).when($\"state\" === \"OK\", 7).when($\"state\" === \"OH\", 6)\n .otherwise(1))\n .withColumn(\"c\", when($\"state\" === \"AK\", \"red\").when($\"state\" === \"TX\", \"orange\").when($\"state\" === \"CA\", \"blue\").when($\"state\" === \"OK\", \"green\").when($\"state\" === \"OH\", \"yellow\")\n .otherwise(\"white\"))\nGeoPointsChart(airportsDFWithStyles, latLonFields=Some((\"lat\", \"long\")), rField = Some(\"r\"), colorField = Some(\"c\"))",
469505
"outputs" : [ ]
506+
}, {
507+
"metadata" : {
508+
"id" : "B4273E6F1E8A4ADA8866AB9A067533E4"
509+
},
510+
"cell_type" : "markdown",
511+
"source" : "---\n## GeoChart"
512+
}, {
513+
"metadata" : {
514+
"trusted" : true,
515+
"input_collapsed" : false,
516+
"collapsed" : true,
517+
"id" : "ACD54EC9C7414EE98C568AD260A377ED"
518+
},
519+
"cell_type" : "code",
520+
"source" : ":sh wget http://data.cyc.opendata.arcgis.com/datasets/57fa576e5e8149b0b744f768e01e5ce1_0.geojson -O Parks_and_Gardens.geojson",
521+
"outputs" : [ ]
522+
}, {
523+
"metadata" : {
524+
"trusted" : true,
525+
"input_collapsed" : false,
526+
"collapsed" : true,
527+
"id" : "750EC1E117C24C28851D00C89298F5A8"
528+
},
529+
"cell_type" : "code",
530+
"source" : "val ut = new Utils(){}\nval geoJSONRepr = ut.parseGeoJSON(scala.io.Source.fromFile(\"Parks_and_Gardens.geojson\").getLines.mkString(\"\"))",
531+
"outputs" : [ ]
532+
}, {
533+
"metadata" : {
534+
"trusted" : true,
535+
"input_collapsed" : false,
536+
"collapsed" : true,
537+
"id" : "012789D6569C4DC6A15579070B0B17B6"
538+
},
539+
"cell_type" : "code",
540+
"source" : ":sh wget http://data.cyc.opendata.arcgis.com/datasets/9b212b7af275438ca9088ff868bda139_9.geojson -O airqual.geojson",
541+
"outputs" : [ ]
542+
}, {
543+
"metadata" : {
544+
"trusted" : true,
545+
"input_collapsed" : false,
546+
"collapsed" : true,
547+
"id" : "E5314FF973454E0B96877C2A6690B914"
548+
},
549+
"cell_type" : "code",
550+
"source" : "val ng = ut.parseGeoJSON(scala.io.Source.fromFile(\"airqual.geojson\").getLines.mkString(\"\"))",
551+
"outputs" : [ ]
552+
}, {
553+
"metadata" : {
554+
"trusted" : true,
555+
"input_collapsed" : false,
556+
"collapsed" : true,
557+
"id" : "273E8A681B4942838D2E0DB37C215F9E"
558+
},
559+
"cell_type" : "code",
560+
"source" : "val gc = GeoChart(Seq(geoJSONRepr))\ngc",
561+
"outputs" : [ ]
562+
}, {
563+
"metadata" : {
564+
"trusted" : true,
565+
"input_collapsed" : false,
566+
"collapsed" : true,
567+
"id" : "1E08A11C501C4E618FDFD7E97F340D79"
568+
},
569+
"cell_type" : "code",
570+
"source" : "gc.addAndApply(Seq(ng))",
571+
"outputs" : [ ]
470572
}, {
471573
"metadata" : {
472574
"id" : "2DDE82591B3E4DE1B09C71CC9292E148"
473575
},
474576
"cell_type" : "markdown",
475-
"source" : "## Fancy charts"
577+
"source" : "---\n## Fancy charts"
476578
}, {
477579
"metadata" : {
478580
"id" : "415502CC4E5E44D7BF7F08978A4C80BD"
@@ -561,7 +663,7 @@
561663
"id" : "7C30152109AD4727827599D6949592F0"
562664
},
563665
"cell_type" : "markdown",
564-
"source" : "## Everything is Dynamic and Reactive"
666+
"source" : "---\n## Everything is Dynamic and Reactive"
565667
}, {
566668
"metadata" : {
567669
"id" : "3AADA3697ABE44BF85FFB47209E06036"
@@ -608,24 +710,14 @@
608710
"id" : "C3A5386580284B5D9C8D4BE907F6FA8C"
609711
},
610712
"cell_type" : "code",
611-
"source" : "val rteam = shuffle(team.toList).take(5)\nval dd = new DropDown(\"All\" :: rteam.map(_.Player))\nval rc = RadarChart(rteam, labelField=Some(\"Player\"), sizes=(800, 600))\nval bout = out\ndd ++ bout ++ rc",
612-
"outputs" : [ ]
613-
}, {
614-
"metadata" : {
615-
"trusted" : true,
616-
"input_collapsed" : false,
617-
"collapsed" : true,
618-
"id" : "C46F272CFC8C476FB022FD2BD6C84811"
619-
},
620-
"cell_type" : "code",
621-
"source" : "dd.selected --> Connection.fromObserver { p =>\n bout(p + \" is selected\")\n rc.applyOn(rc.originalData.filter(_.Player == p || p == \"All\"))\n}",
713+
"source" : "val rteam = shuffle(team.toList).take(5)\nval dd = new DropDown(\"All\" :: rteam.map(_.Player))\nval rc = RadarChart(rteam, labelField=Some(\"Player\"), sizes=(800, 600))\nval bout = out\n\ndd.selected --> Connection.fromObserver { p =>\n bout(p + \" is selected\")\n rc.applyOn(rc.originalData.filter(_.Player == p || p == \"All\"))\n}\n\ndd ++ bout ++ rc",
622714
"outputs" : [ ]
623715
}, {
624716
"metadata" : {
625717
"id" : "42D57AE9A76F4FA1A114BA95C2E1FADB"
626718
},
627719
"cell_type" : "markdown",
628-
"source" : "## Synchronization"
720+
"source" : "---\n## Synchronization"
629721
}, {
630722
"metadata" : {
631723
"id" : "7460856EFF64425A83DD0DD4B5814360"
@@ -637,7 +729,7 @@
637729
"id" : "593AD5B17B854593B20EA88F8B18E338"
638730
},
639731
"cell_type" : "markdown",
640-
"source" : "## Create new chart type... live"
732+
"source" : "---\n## Create new chart type... live"
641733
}, {
642734
"metadata" : {
643735
"id" : "E51CB610568E4936ADA024549386E8E3"
@@ -723,7 +815,7 @@
723815
"id" : "787984960C10439D81F5DA7FBD00505A"
724816
},
725817
"cell_type" : "markdown",
726-
"source" : "## Contexts with interpolation"
818+
"source" : "---\n## Contexts with interpolation"
727819
}, {
728820
"metadata" : {
729821
"trusted" : true,
@@ -742,7 +834,7 @@
742834
"id" : "C924A6AA52B94822B64718240069CFB0"
743835
},
744836
"cell_type" : "code",
745-
"source" : "val ok = \"snb\"",
837+
"source" : "val ok = \"$\\\\LaTeX$ interpolated in Scala is $\\\\Re$\"",
746838
"outputs" : [ ]
747839
}, {
748840
"metadata" : {
@@ -752,7 +844,7 @@
752844
"id" : "416D1E2D54154AE7845CE8FC14ECF334"
753845
},
754846
"cell_type" : "code",
755-
"source" : ":markdown \nWe're using a **$ok** file",
847+
"source" : ":markdown \nYup, **$ok** in Spark Notebook",
756848
"outputs" : [ ]
757849
}, {
758850
"metadata" : {
@@ -762,7 +854,7 @@
762854
"id" : "7B10ABEE96784E458F34447A8C9E7C46"
763855
},
764856
"cell_type" : "code",
765-
"source" : ":javascript\nalert(\"$ok\")",
857+
"source" : ":javascript\nalert(\"I am ${(\"whoami\".!!).trim}\")",
766858
"outputs" : [ ]
767859
} ],
768860
"nbformat" : 4

0 commit comments

Comments
 (0)