SuccessChanges

Summary

  1. Filtering Cartesian RDD (details)
  2. Vector Join using FilteringCartesianRDD (details)
  3. Do Not Bring Metadata To Driver (details)
  4. Push More Computation Down Into RDD (details)
  5. Add Attribution (details)
  6. Remove Naive VectorJoin (details)
  7. Broaden Tests To Account For Partitioning (details)
  8. Pull Meta-RDD Creation Into FilteredCartesianRDD (details)
Commit 02e8d67ddd9f7d32d300b92773bfeeb844e83873 by James McClain
Filtering Cartesian RDD

This is largely identical to the CartesianRDD type provided by Spark,
except that it accepts a predicate which allows non-viable pairs to be
not generated.  This can reduce computation and communication.

In the case of VectorJoin, items in partitions with disjoint envelopes
are never generated.
The file was addedspark/src/main/scala/org/apache/spark/rdd/FilteredCartesianRDD.scala
Commit eb3d5e2cf066e0c188218ae75bc0a339506caf62 by James McClain
Vector Join using FilteringCartesianRDD
The file was modifiedspark/src/main/scala/geotrellis/spark/join/VectorJoin.scala (diff)
Commit 9a38c03384e1ab5a653bec1ecdf3fc91f7a7a6ca by James McClain
Do Not Bring Metadata To Driver
The file was modifiedspark/src/main/scala/org/apache/spark/rdd/FilteredCartesianRDD.scala (diff)
The file was modifiedspark/src/main/scala/geotrellis/spark/join/VectorJoin.scala (diff)
Commit 383327288905e7dd39c4f7d7b38f7d61c7b7bc3f by James McClain
Push More Computation Down Into RDD
The file was modifiedspark/src/main/scala/org/apache/spark/rdd/FilteredCartesianRDD.scala (diff)
The file was modifiedspark/src/main/scala/geotrellis/spark/join/VectorJoin.scala (diff)
The file was modifiedspark/src/main/scala/org/apache/spark/rdd/FilteredCartesianRDD.scala (diff)
The file was modifiedspark/src/main/scala/geotrellis/spark/join/VectorJoin.scala (diff)
Commit fcdc7337ecd63a85893aeb80081302bc867e061a by James McClain
Broaden Tests To Account For Partitioning
The file was modifiedspark/src/test/scala/geotrellis/spark/join/VectorJoinRDDSpec.scala (diff)
Commit e231e52d605539a0ed3f9b9ced3404a5db5af0e5 by James McClain
Pull Meta-RDD Creation Into FilteredCartesianRDD

The motivation for this is an API issue: the two Meta-RDDs must be
created by performing the mapPartitions operation on the two input RDDS.
This change gurantees that.
The file was modifiedspark/src/main/scala/org/apache/spark/rdd/FilteredCartesianRDD.scala (diff)
The file was modifiedspark/src/main/scala/geotrellis/spark/join/VectorJoin.scala (diff)