Commits

Breadcrumbs

History for

onmaster

User selector

All users

All time

Commit History

Commits on Aug 9, 2019

• [SPARK-27371][CORE] Support GPU-aware resources scheduling in Standalone

  Show description for cbad616
  

  Ngone51

  authored and

  tgravescs

  committed
  cbad616

  Copy full SHA for cbad616

• [SPARK-28651][SS] Force the schema of Streaming file source to be nullable

  Show description for 5bb6994
  

  zsxwing

  authored and

  HyukjinKwon

  committed
  5bb6994

  Copy full SHA for 5bb6994

Commits on Aug 8, 2019

• [SPARK-28454][PYTHON] Validate LongType in `createDataFrame(verifySchema=True)`

  Show description for bda5b51
  

  simplylizz

  authored and

  HyukjinKwon

  committed
  bda5b51

  Copy full SHA for bda5b51

Commits on Aug 5, 2019

• [SPARK-28486][CORE][PYTHON] Map PythonBroadcast's data file to a BroadcastBlock to avoid delete by GC

  Show description for 94499af
  

  Ngone51

  authored and

  HyukjinKwon

  committed
  94499af

  Copy full SHA for 94499af

Commits on Aug 3, 2019

• [SPARK-28582][PYTHON] Fix flaky test DaemonTests.do_termination_test which fail on Python 3.7

  Show description for b3394db
  

  WeichenXu123

  authored and

  HyukjinKwon

  committed
  b3394db

  Copy full SHA for b3394db

Commits on Aug 2, 2019

• Revert "[SPARK-28582][PYSPARK] Fix flaky test DaemonTests.do_termination_test which fail on Python 3.7"

  Show description for 8ae032d
  dongjoon-hyun

  committed
  8ae032d

  Copy full SHA for 8ae032d

• [SPARK-23469][ML] HashingTF should use corrected MurmurHash3 implementation

  Show description for 660423d
  

  huaxingao

  authored and

  srowen

  committed
  660423d

  Copy full SHA for 660423d

• [SPARK-28582][PYSPARK] Fix flaky test DaemonTests.do_termination_test which fail on Python 3.7

  Show description for fbeee0c
  

  WeichenXu123

  authored and

  HyukjinKwon

  committed
  fbeee0c

  Copy full SHA for fbeee0c

Commits on Jul 31, 2019

• [SPARK-28153][PYTHON] Use AtomicReference at InputFileBlockHolder (to support input_file_name with Python UDF)

  Show description for b8e13b0
  

  HyukjinKwon

  authored and

  cloud-fan

  committed
  b8e13b0

  Copy full SHA for b8e13b0

• [SPARK-25382][SQL][PYSPARK] Remove ImageSchema.readImages in 3.0

  Show description for a745381
  

  WeichenXu123

  authored and

  HyukjinKwon

  committed
  a745381

  Copy full SHA for a745381

• [SPARK-26175][PYTHON] Redirect the standard input of the forked child to devnull in daemon

  Show description for 3b14088
  

  WeichenXu123

  authored and

  HyukjinKwon

  committed
  3b14088

  Copy full SHA for 3b14088

Commits on Jul 30, 2019

• [SPARK-28399][ML][PYTHON] implement RobustScaler

  Show description for 44c28d7
  

  zhengruifeng

  authored and

  srowen

  committed
  44c28d7

  Copy full SHA for 44c28d7

Commits on Jul 29, 2019

• [SPARK-28471][SQL] Replace `yyyy` by `uuuu` in date-timestamp patterns without era

  Show description for a5a5da7
  

  MaxGekk

  authored and

  dongjoon-hyun

  committed
  a5a5da7

  Copy full SHA for a5a5da7

Commits on Jul 28, 2019

• [SPARK-21481][ML] Add indexOf method in ml.feature.HashingTF

  Show description for 70f82fd
  

  huaxingao

  authored and

  srowen

  committed
  70f82fd

  Copy full SHA for 70f82fd

Commits on Jul 27, 2019

• [MINOR][PYTHON] Use `_memory_limit` to get worker memory conf in `rdd.py`

  Show description for fbaa177
  

  WangGuangxin

  authored and

  dongjoon-hyun

  committed
  fbaa177

  Copy full SHA for fbaa177

Commits on Jul 26, 2019

• [SPARK-28507][ML][PYSPARK] Remove deprecated API context(self, sqlContext) from pyspark/ml/util.py

  Show description for 3de4e1b
  

  huaxingao

  authored and

  srowen

  committed
  3de4e1b

  Copy full SHA for 3de4e1b

Commits on Jul 20, 2019

• [SPARK-28243][PYSPARK][ML] Remove setFeatureSubsetStrategy and setSubsamplingRate from Python TreeEnsembleParams

  Show description for 72c80ee
  

  huaxingao

  authored and

  srowen

  committed
  72c80ee

  Copy full SHA for 72c80ee

Commits on Jul 18, 2019

• [SPARK-28439][PYTHON][SQL] Add support for count: Column in array_repeat

  Show description for a0c2fa6
  

  zero323

  authored and

  dongjoon-hyun

  committed
  a0c2fa6

  Copy full SHA for a0c2fa6

• [SPARK-28411][PYTHON][SQL] InsertInto with overwrite is not honored

  Show description for 971e832
  

  huaxingao

  authored and

  HyukjinKwon

  committed
  971e832

  Copy full SHA for 971e832

• [SPARK-27609][PYTHON] Convert values of function options to strings

  Show description for 70073b1
  

  MaxGekk

  authored and

  HyukjinKwon

  committed
  70073b1

  Copy full SHA for 70073b1

Commits on Jul 17, 2019

• [SPARK-28418][PYTHON][SQL] Wait for event process in 'test_query_execution_listener_on_collect'

  Show description for 66179fa
  HyukjinKwon

  committed
  66179fa

  Copy full SHA for 66179fa

Commits on Jul 15, 2019

• [SPARK-28381][PYSPARK] Upgraded version of Pyrolite to 4.30

  Show description for 591de42
  

  viirya

  authored and

  HyukjinKwon

  committed
  591de42

  Copy full SHA for 591de42

Commits on Jul 14, 2019

• [SPARK-28378][PYTHON] Remove usage of cgi.escape

  Show description for 707411f
  

  viirya

  authored and

  HyukjinKwon

  committed
  707411f

  Copy full SHA for 707411f

Commits on Jul 13, 2019

• [SPARK-28355][CORE][PYTHON] Use Spark conf for threshold at which command is compressed by broadcast

  Show description for 79e2047
  

  jessecai

  authored and

  gatorsmile

  committed
  79e2047

  Copy full SHA for 79e2047

Commits on Jul 11, 2019

• [SPARK-28234][CORE][PYTHON] Add python and JavaSparkContext support to get resources

  Show description for f84cca2
  

  tgravescs

  authored and

  HyukjinKwon

  committed
  f84cca2

  Copy full SHA for f84cca2

Commits on Jul 10, 2019

• [SPARK-28323][SQL][PYTHON] PythonUDF should be able to use in join condition

  Show description for 7858e53
  

  viirya

  authored and

  BryanCutler

  committed
  7858e53

  Copy full SHA for 7858e53

Commits on Jul 9, 2019

• [SPARK-28140][MLLIB][PYTHON] Accept DataFrames in RowMatrix and IndexedRowMatrix constructors

  Show description for a32c92c
  

  henrydavidge

  authored and

  srowen

  committed
  a32c92c

  Copy full SHA for a32c92c

• [SPARK-28273][SQL][PYTHON] Convert and port 'pgSQL/case.sql' into UDF test base

  Show description for fe3e34d
  

  HyukjinKwon

  authored and

  cloud-fan

  committed
  fe3e34d

  Copy full SHA for fe3e34d

Commits on Jul 7, 2019

• [SPARK-28226][PYTHON] Document Pandas UDF mapInPandas

  Show description for cdbc302
  HyukjinKwon

  committed
  cdbc302

  Copy full SHA for cdbc302

Commits on Jul 5, 2019

• [SPARK-28206][PYTHON] Remove the legacy Epydoc in PySpark API documentation

  Show description for fe75ff8
  

  HyukjinKwon

  authored and

  mengxr

  committed
  fe75ff8

  Copy full SHA for fe75ff8

• [SPARK-28198][PYTHON][FOLLOW-UP] Rename mapPartitionsInPandas to mapInPandas with a separate evaluation type

  Show description for 5c55812
  HyukjinKwon

  committed
  5c55812

  Copy full SHA for 5c55812

Commits on Jul 3, 2019

• [SPARK-28240][PYTHON] Fix Arrow tests to pass with Python 2.7 and latest PyArrow and Pandas in PySpark

  Show description for 5f7aceb
  HyukjinKwon

  committed
  5f7aceb

  Copy full SHA for 5f7aceb

Commits on Jul 2, 2019

• [SPARK-28198][PYTHON] Add mapPartitionsInPandas to allow an iterator of DataFrames

  Show description for 02f4763
  HyukjinKwon

  committed
  02f4763

  Copy full SHA for 02f4763

Commits on Jul 1, 2019

• [SPARK-28170][ML][PYTHON] Uniform Vectors and Matrix documentation

  Show description for 048224c
  

  mgaido91

  authored and

  HyukjinKwon

  committed
  048224c

  Copy full SHA for 048224c

Commits on Jun 28, 2019

• [SPARK-28056][.2][PYTHON][SQL] add docstring/doctest for SCALAR_ITER Pandas UDF

  Show description for 8299600
  

  mengxr

  authored and

  gatorsmile

  committed
  8299600

  Copy full SHA for 8299600