如何在PySpark中指定镶木地板块大小和页面大小？我到处搜索,但是找不到函数调用或导入库的任何文档.

解决方法:

根据spark-user archives

sc.hadoopConfiguration.setInt("dfs.blocksize", some_value)
sc.hadoopConfiguration.setInt("parquet.block.size", some_value)

所以在PySpark

sc._jsc.hadoopConfiguration().setInt("dfs.blocksize", some_value)
sc._jsc.hadoopConfiguration().setInt("parquet.block.size", some_value)