Skip to content

[DOC] Minor modification to Streaming docs with regards to parallel data receiving #6544

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
nishkamravi2 wants to merge 69 commits into from
Closed

[DOC] Minor modification to Streaming docs with regards to parallel data receiving #6544

Changes from 1 commit
Commits
Show all changes
69 commits
Select commit Hold shift + click to select a range
681b36f
Fix for SPARK-1758: failing test org.apache.spark.JavaAPISuite.wholeT…
nishkamravi2 May 8, 2014
5108700
Fix in Spark for the Concurrent thread modification issue (SPARK-1097…
nishkamravi2 Jun 3, 2014
6b840f0
Undo the fix for SPARK-1758 (the problem is fixed)
nishkamravi2 Jun 3, 2014
df2aeb1
Improved fix for ConcurrentModificationIssue (Spark-1097, Hadoop-10456)
nishkamravi2 Jun 9, 2014
eb663ca
Merge branch 'master' of https://github.com/apache/spark
nishkamravi2 Jun 9, 2014
5423a03
Merge branch 'master' of https://github.com/apache/spark
nishkamravi2 Jun 10, 2014
3bf8fad
Merge branch 'master' of https://github.com/apache/spark
nishkamravi2 Jun 16, 2014
2b630f9
Accept memory input as "30g", "512M" instead of an int value, to be c…
nishkamravi2 Jun 16, 2014
efd688a
Merge branch 'master' of https://github.com/apache/spark
nishkamravi2 Jul 13, 2014
2e69f11
Merge branch 'master' of https://github.com/apache/spark into master_…
nishkamravi2 Sep 21, 2014
ebcde10
Modify default YARN memory_overhead-- from an additive constant to a …
nishkamravi2 Sep 22, 2014
1cf2d1e
Update YarnAllocator.scala
nishkamravi2 Sep 22, 2014
f00fa31
Improving logging for AM memoryOverhead
nishkamravi2 Sep 22, 2014
c726bd9
Merge branch 'master' of https://github.com/apache/spark into master_…
nishkamravi2 Sep 25, 2014
362da5e
Additional changes for yarn memory overhead
nishkamravi2 Sep 25, 2014
42c2c3d
Additional changes for yarn memory overhead issue
nishkamravi2 Sep 25, 2014
dac1047
Additional documentation for yarn memory overhead issue
nishkamravi2 Sep 25, 2014
5ac2ec1
Remove out
nishkamravi2 Sep 25, 2014
35daa64
Slight change in the doc for yarn memory overhead
nishkamravi2 Sep 25, 2014
8f76c8b
Doc change for yarn memory overhead
nishkamravi2 Sep 25, 2014
636a9ff
Update YarnAllocator.scala
nishkamravi2 Sep 30, 2014
5f8f9ed
Merge branch 'master' of https://github.com/apache/spark into master_…
nishkamravi2 Nov 19, 2014
9f6583e
Changes to maxResultSize code (improve error message and add conditio…
nishkamravi2 Nov 19, 2014
3e1b616
Modify test for maxResultSize
nishkamravi2 Nov 19, 2014
535295a
Update TaskSetManager.scala
nishkamravi2 Nov 19, 2014
5c9a4cb
Update TaskSetManagerSuite.scala
nishkamravi2 Nov 19, 2014
b446edc
Merge branch 'master' of https://github.com/apache/spark into master_…
nishkamravi2 Nov 19, 2014
79ea8b4
Merge branch 'master' of https://github.com/apache/spark into master_…
nishkamravi2 Feb 19, 2015
f0d12de
Workaround for IllegalStateException caused by recent changes to Bloc…
nishkamravi2 Feb 19, 2015
3c5ddba
Update DiskBlockManager.scala
nishkamravi2 Feb 19, 2015
494d8c0
Update DiskBlockManager.scala
nishkamravi2 Feb 19, 2015
71d0e17
Merge branch 'master' of https://github.com/apache/spark into master_…
nishkamravi2 Feb 20, 2015
ba1eb8b
Try-catch block around the two occurrences of removeShutDownHook. Del…
nishkamravi2 Feb 20, 2015
f7e33c2
Merge branch 'master_nravi' of https://github.com/nishkamravi2/spark …
nishkamravi2 Feb 20, 2015
0ce2c32
Update HadoopRDD.scala
nishkamravi2 Feb 20, 2015
6f41a1d
Update NewHadoopRDD.scala
nishkamravi2 Feb 20, 2015
d453197
Update NewHadoopRDD.scala
nishkamravi2 Feb 20, 2015
ee902fa
Merge branch 'master' of https://github.com/apache/spark into master_…
nishkamravi2 Mar 18, 2015
bfe96e0
Merge branch 'master' of https://github.com/apache/spark into master_…
nishkamravi2 Mar 19, 2015
746f35b
"hadoop" string in the assembly name should not be mandatory (everywh…
nishkamravi2 Mar 19, 2015
4bd4489
Update AbstractCommandBuilder.java
nishkamravi2 Mar 19, 2015
32c3ab3
Update AbstractCommandBuilder.java
nishkamravi2 Mar 19, 2015
35af990
Merge branch 'master' of https://github.com/apache/spark into master_…
nishkamravi2 Mar 20, 2015
06bfeb0
Update spark-class
nishkamravi2 Mar 20, 2015
ac58975
spark-class changes
nishkamravi2 Mar 20, 2015
345206a
spark-class merge
nishkamravi2 Mar 20, 2015
3faa7a4
Launcher library changes (SPARK-6406)
nishkamravi2 Mar 23, 2015
ccdc334
Merge branch 'master' of https://github.com/apache/spark into master_…
nishkamravi2 Mar 23, 2015
d9658d6
Changes for SPARK-6406
nishkamravi2 Mar 24, 2015
458af39
Locate the jar using getLocation, obviates the need to pass assembly …
nishkamravi2 Mar 24, 2015
4a45d6a
Merge branch 'master' of https://github.com/apache/spark into master_…
nishkamravi2 Mar 24, 2015
b7f4ae7
Merge branch 'master' of https://github.com/apache/spark into master_…
nishkamravi2 Mar 27, 2015
36a6f87
Minor changes and bug fixes
nishkamravi2 Mar 27, 2015
bad4349
Update Main.java
nishkamravi2 Mar 27, 2015
1c13b79
Merge branch 'master' of https://github.com/apache/spark into master_…
nishkamravi2 Apr 16, 2015
2be1e76
Merge branch 'master' of https://github.com/apache/spark into master_…
nishkamravi2 Apr 24, 2015
8691e0c
Add a try/catch block around Utils.removeShutdownHook
nishkamravi2 Apr 24, 2015
97c383e
Update Utils.scala
nishkamravi2 Apr 24, 2015
474e3bf
Update DiskBlockManager.scala
nishkamravi2 Apr 24, 2015
0f1abd0
Update Utils.scala
nishkamravi2 Apr 24, 2015
1b81383
Merge branch 'master' of https://github.com/apache/spark into master_…
nishkamravi2 Apr 28, 2015
981afd2
Check for read permission before initiating copy
nishkamravi2 Apr 28, 2015
eaa13b5
Update Client.scala
nishkamravi2 Apr 29, 2015
059ce82
Merge branch 'master' of https://github.com/apache/spark into master_…
nishkamravi2 May 15, 2015
5190f79
In translating from deprecated spark.kryoserializer.buffer.mb to spar…
nishkamravi2 May 15, 2015
5261bf6
Add a test for deprecated config spark.kryoserializer.buffer.mb
nishkamravi2 May 15, 2015
171a53c
Update SparkConfSuite.scala
nishkamravi2 May 15, 2015
9e59c71
Merge branch 'master' of https://github.com/apache/spark into master_…
nishkamravi2 Jun 1, 2015
46e8c03
Slight modification to streaming docs
nishkamravi2 Jun 1, 2015
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Update HadoopRDD.scala
  • Loading branch information
@nishkamravi2
nishkamravi2 committed Feb 20, 2015
commit 0ce2c32d22f65b485ee506e095da25b08a8c317d
7 changes: 5 additions & 2 deletions core/src/main/scala/org/apache/spark/rdd/HadoopRDD.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -270,8 +270,11 @@ class HadoopRDD[K, V](
}
}
} catch {
case e: Exception =>
logWarning("Exception in RecordReader.close()", e)
case e: Exception => {
if (!Utils.inShutdown()) {
logWarning("Exception in RecordReader.close()", e)
}
}
}
}
}
Expand Down