# Index

### Big Data Systems

* [**MapReduce: Simplified Data Processing on Large Clusters**](https://static.googleusercontent.com/media/research.google.com/en/archive/mapreduce-osdi04.pdf) - Dean et al., OSDI '04 \[[Summary](https://xzhu0027.gitbook.io/blog/big-data-sys/mapreduce-simplified-data-processing-on-large-clusters)]
* [**Dryad: Distributed Data-Parallel Programs from Sequential Building Blocks**](https://www.microsoft.com/en-us/research/wp-content/uploads/2007/03/eurosys07.pdf) - Isard et al., EuroSys '07 \[[Summary](https://xzhu0027.gitbook.io/blog/big-data-sys/dryad-distributed-data-parallel-programs-from-sequentialbuilding-blocks)]
* [**MapReduce Online**](http://www.neilconway.org/docs/nsdi2010_hop.pdf) - Condie et al., NSDI '10 \[[Summary](https://xzhu0027.gitbook.io/blog/big-data-sys/mapreduce-online)]
* [**Resilient Distributed Datasets: A Fault-Tolerant Abstraction for In-Memory Cluster Computing**](https://www.usenix.org/system/files/conference/nsdi12/nsdi12-final138.pdf)(Spark) - Zaharia et al., NSDI '12 \[[Summary](https://xzhu0027.gitbook.io/blog/big-data-sys/resilient-distributed-datasets-a-fault-tolerant-abstraction-for-in-memory-cluster-computing)]
* [**Naiad: A Timely Dataflow System**](http://sigops.org/s/conferences/sosp/2013/papers/p439-murray.pdf) - Murray et al., SOSP '13 \[[Summary](https://xzhu0027.gitbook.io/blog/big-data-sys/naiad-a-timely-dataflow-system)]
* [**Discretized Streams: Fault-Tolerant Streaming Computation at Scale**](https://people.csail.mit.edu/matei/papers/2013/sosp_spark_streaming.pdf) - Zaharia et al., SOSP '13 \[[Summary](https://xzhu0027.gitbook.io/blog/big-data-sys/discretized-streams-fault-tolerant-streaming-computation-at-scale)]
* [**The Dataflow Model: A Practical Approach to Balancing Correctness, Latency, and Cost in Massive-Scale** ](https://www.vldb.org/pvldb/vol8/p1792-Akidau.pdf) - Akidau et al., VLDB '15 \[[Summary](https://xzhu0027.gitbook.io/blog/big-data-sys/the-dataflow-model-a-practical-approach-to-balancing-correctness-latency-and-cost-in-massive-scale)]

### [Resource Management ](https://xzhu0027.gitbook.io/blog/big-data-sys/resource-management)

* [**Mesos: A Platform for Fine-Grained Resource Sharing in the Data Center**](https://people.eecs.berkeley.edu/~alig/papers/mesos.pdf) - Hindman et al., NSDI '11 \[[Summary](https://xzhu0027.gitbook.io/blog/big-data-sys/mesos-a-platform-for-fine-grained-resource-sharing-in-the-data-center)]
* [**Dominant Resource Fairness** ](https://cs.stanford.edu/~matei/papers/2011/nsdi_drf.pdf) - Ghodsi et al., NSDI '11\[Summary]
* [**PACMan: Coordinated Memory Caching for Parallel Jobs**](https://www.usenix.org/system/files/conference/nsdi12/pacman.pdf) - Ananthanarayanan et al., NSDI '12 \[[Summary](https://xzhu0027.gitbook.io/blog/big-data-sys/pacman-coordinated-memory-caching-for-parallel-jobs)]
* [**Apache Hadoop YARN: Yet Another Resource Negotiator**](https://www.cse.ust.hk/~weiwa/teaching/Fall15-COMP6611B/reading_list/YARN.pdf) - Vavilapallih et al., SoCC '13 \[[Summary](https://xzhu0027.gitbook.io/blog/big-data-sys/apache-hadoop-yarn-yet-another-resource-negotiator)]
* [**Making Sense of Performance in Data Analytics Framework** ](https://www.usenix.org/system/files/conference/nsdi15/nsdi15-paper-ousterhout.pdf)- Ousterhout et al., SOSP '15 \[[Summary](https://xzhu0027.gitbook.io/blog/big-data-sys/making-sense-of-performance-in-data-analytics-framework)]
* [**Large-scale cluster management at Google with Borg**](https://storage.googleapis.com/pub-tools-public-publication-data/pdf/43438.pdf) **-** Verma et al., EuroSys '15 \[[Summary](https://xzhu0027.gitbook.io/blog/big-data-sys/large-scale-cluster-management-at-google-with-borg)]
* [**Efficient Queue Management for Cluster Scheduling**](https://www.cse.ust.hk/~weiwa/teaching/Fall16-COMP6611B/reading_list/Yaq.pdf) - Rasley et al., EuroSys '16 \[[Summary](https://xzhu0027.gitbook.io/blog/big-data-sys/other-interesting-papers#efficient-queue-management-for-cluster-scheduling-rasley-et-al-eurosys-16)]
* [**Morpheus: Towards Automated SLOs for Enterprise Clusters**](https://www.usenix.org/conference/osdi16/technical-sessions/presentation/jyothi) -Jyothi et al., OSDI '16
* [**Borg, Omega, and Kubernetes**](https://ai.google/research/pubs/pub44843) - Burns et al., ACM Queue '16
* [**Learning Scheduling Algorithms for Data Processing Clusters**](https://web.mit.edu/decima/content/sigcomm-2019.pdf) - Hongzi et al., SIGCOMM '19 

### Scheduling 

* [**Delay Scheduling: A Simple Technique for Achieving Locality and Fairness in Cluster Scheduling**](https://cs.stanford.edu/~matei/papers/2010/eurosys_delay_scheduling.pdf) - Zaharia et al., EuroSys '10 \[[Summary](https://xzhu0027.gitbook.io/blog/big-data-sys/delay-scheduling-a-simple-technique-for-achieving-locality-and-fairness-in-cluster-scheduling)] 
* [**Sparrow: Distributed, Low Latency Scheduling**](https://cs.stanford.edu/~matei/papers/2013/sosp_sparrow.pdf) - Ousterhout et al., SOSP '13 \[[Summary](https://xzhu0027.gitbook.io/blog/big-data-sys/sparrow-distributed-low-latency-scheduling)]
* [**Multi-Resource Packing for Cluster Schedulers**](https://www.cs.cmu.edu/~xia/resources/Documents/grandl_sigcomm14.pdf) - Grandl et al., SIGCOMM '14 \[[Summary](https://xzhu0027.gitbook.io/blog/big-data-sys/multi-resource-packing-for-cluster-schedulers)]
* [**CLARINET: WAN-Aware Optimization for Analytics Queries**](https://www.usenix.org/system/files/conference/osdi16/osdi16-viswanathan.pdf) **-** Viswanathan et al., OSDI '16 \[[Summary](https://xzhu0027.gitbook.io/blog/big-data-sys/clarinet-wan-aware-optimization-for-analytics-queries)]
* [**Monotasks: Architecting for Performance Clarity in Data Analytics Frameworks**](http://kayousterhout.org/publications/sosp17-final183.pdf)  - Ousterhout et al., SOSP '17 \[[Summary](https://xzhu0027.gitbook.io/blog/big-data-sys/monotasks-architecting-for-performance-clarity-in-data-analytics-frameworks)]
* [**Drizzle: Fast and Adaptable Stream Processing at Scale** ](http://shivaram.org/publications/drizzle-sosp17.pdf) **-** Venkataraman et al., SOSP '17 \[[Summary](https://xzhu0027.gitbook.io/blog/big-data-sys/drizzle-fast-and-adaptable-stream-processing-at-scale)]

### Storage

* [**Flat Datacenter Storage**](https://www.usenix.org/system/files/conference/osdi12/osdi12-final-75.pdf) - Nightingale et al., OSDI '12 \[[Summary](https://xzhu0027.gitbook.io/blog/distributed-storage/flat-datacenter-storage)]
* [**Tachyon: Reliable, Memory Speed Storage for Cluster Computing Frameworks**](https://people.csail.mit.edu/matei/papers/2014/socc_tachyon.pdf) - Li et al., SoCC '14 \[[Summary](https://xzhu0027.gitbook.io/blog/distributed-storage/tachyon-reliable-memory-speed-storage-for-cluster-computing-frameworks)]
* [**EC-Cache: Load-balanced, Low-latency Cluster Caching with Online Erasure Coding**](https://www.usenix.org/system/files/conference/osdi16/osdi16-rashmi.pdf) - Rashmi et al., OSDI '16

### Fault Tolerance 

* [**Improving MapReduce Performance in Heterogeneous Environments**](http://courses.cs.vt.edu/cs5204/fall12-kafura/Papers/MapReduce/Map-Reduce-Hadoop.pdf) **-** Zaharia et al., OSDI '08 \[[Summary](https://xzhu0027.gitbook.io/blog/big-data-sys/improving-mapreduce-performance-in-heterogeneous-environments)]
* [**Reining in the Outliers in Map-Reduce Clusters using Mantri** ](https://www.usenix.org/legacy/events/osdi10/tech/full_papers/Ananthanarayanan.pdf)**-** Ananthanarayanan et al., OSDI '10 \[[Summary](https://xzhu0027.gitbook.io/blog/big-data-sys/reining-in-the-outliers-in-map-reduce-clusters-using-mantri)]
* [**Effective Straggler Mitigation: Attack of the Clones** ](https://www.usenix.org/system/files/conference/nsdi13/nsdi13-final231.pdf) **-** Ananthanarayanan et al., NSDI '13 \[[Summary](https://xzhu0027.gitbook.io/blog/big-data-sys/effective-straggler-mitigation-attack-of-the-clones)]
* [**Interruptible Tasks: Treating Memory Pressure AsInterrupts for Highly Scalable Data-Parallel Program**](https://people.cs.uchicago.edu/~shanlu/paper/sosp15-itask.pdf) - Fang et al., SOSP '15 \[[Summary](https://xzhu0027.gitbook.io/blog/big-data-sys/interruptible-tasks-treating-memory-pressure-asinterrupts-for-highly-scalable-data-parallel-progra)]
* [**Lineage Stash: Fault Tolerance Off the Critical Path**](https://dl.acm.org/authorize?N695036) - Wang et al., SOSP '19 \[[Summary](https://xzhu0027.gitbook.io/blog/big-data-sys/other-interesting-papers#lineage-stash-fault-tolerance-off-the-critical-path-wang-et-al-sosp-19)] 

### Misc.

* [**ApproxHadoop: Bringing Approximations to MapReduce Frameworks**](https://dl.acm.org/citation.cfm?id=2694351) - Goiri et al., ASPLOS '15
* [**Yak: A High-Performance Big-Data-Friendly Garbage Collector**](https://www.usenix.org/system/files/conference/osdi16/osdi16-nguyen.pdf) - Nguyen et al., OSDI '16