> For the complete documentation index, see [llms.txt](https://xzhu0027.gitbook.io/blog/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://xzhu0027.gitbook.io/blog/cloud-computing/index.md).

# Index

### Big Data Systems

* [**MapReduce: Simplified Data Processing on Large Clusters**](https://static.googleusercontent.com/media/research.google.com/en//archive/mapreduce-osdi04.pdf) - Dean et al., OSDI '04 \[[Summary](https://xzhu0027.gitbook.io/blog/big-data-sys/mapreduce-simplified-data-processing-on-large-clusters)]
* [**Dryad: Distributed Data-Parallel Programs from Sequential Building Blocks**](https://www.microsoft.com/en-us/research/wp-content/uploads/2007/03/eurosys07.pdf) - Isard et al., EuroSys '07 \[[Summary](https://xzhu0027.gitbook.io/blog/big-data-sys/dryad-distributed-data-parallel-programs-from-sequentialbuilding-blocks)]
* [**MapReduce Online**](http://www.neilconway.org/docs/nsdi2010_hop.pdf) - Condie et al., NSDI '10 \[[Summary](https://xzhu0027.gitbook.io/blog/big-data-sys/mapreduce-online)]
* [**Resilient Distributed Datasets: A Fault-Tolerant Abstraction for In-Memory Cluster Computing**](https://www.usenix.org/system/files/conference/nsdi12/nsdi12-final138.pdf)(Spark) - Zaharia et al., NSDI '12 \[[Summary](https://xzhu0027.gitbook.io/blog/big-data-sys/resilient-distributed-datasets-a-fault-tolerant-abstraction-for-in-memory-cluster-computing)]
* [**Naiad: A Timely Dataflow System**](http://sigops.org/s/conferences/sosp/2013/papers/p439-murray.pdf) - Murray et al., SOSP '13 \[[Summary](https://xzhu0027.gitbook.io/blog/big-data-sys/naiad-a-timely-dataflow-system)]
* [**Discretized Streams: Fault-Tolerant Streaming Computation at Scale**](https://people.csail.mit.edu/matei/papers/2013/sosp_spark_streaming.pdf) - Zaharia et al., SOSP '13 \[[Summary](https://xzhu0027.gitbook.io/blog/big-data-sys/discretized-streams-fault-tolerant-streaming-computation-at-scale)]
* [**The Dataflow Model: A Practical Approach to Balancing Correctness, Latency, and Cost in Massive-Scale** ](https://www.vldb.org/pvldb/vol8/p1792-Akidau.pdf) - Akidau et al., VLDB '15 \[[Summary](https://xzhu0027.gitbook.io/blog/big-data-sys/the-dataflow-model-a-practical-approach-to-balancing-correctness-latency-and-cost-in-massive-scale)]

### [Resource Management ](https://xzhu0027.gitbook.io/blog/big-data-sys/resource-management)

* [**Mesos: A Platform for Fine-Grained Resource Sharing in the Data Center**](https://people.eecs.berkeley.edu/~alig/papers/mesos.pdf) - Hindman et al., NSDI '11 \[[Summary](https://xzhu0027.gitbook.io/blog/big-data-sys/mesos-a-platform-for-fine-grained-resource-sharing-in-the-data-center)]
* [**Dominant Resource Fairness** ](https://cs.stanford.edu/~matei/papers/2011/nsdi_drf.pdf) - Ghodsi et al., NSDI '11\[Summary]
* [**PACMan: Coordinated Memory Caching for Parallel Jobs**](https://www.usenix.org/system/files/conference/nsdi12/pacman.pdf) - Ananthanarayanan et al., NSDI '12 \[[Summary](https://xzhu0027.gitbook.io/blog/big-data-sys/pacman-coordinated-memory-caching-for-parallel-jobs)]
* [**Apache Hadoop YARN: Yet Another Resource Negotiator**](https://www.cse.ust.hk/~weiwa/teaching/Fall15-COMP6611B/reading_list/YARN.pdf) - Vavilapallih et al., SoCC '13 \[[Summary](https://xzhu0027.gitbook.io/blog/big-data-sys/apache-hadoop-yarn-yet-another-resource-negotiator)]
* [**Making Sense of Performance in Data Analytics Framework** ](https://www.usenix.org/system/files/conference/nsdi15/nsdi15-paper-ousterhout.pdf)- Ousterhout et al., SOSP '15 \[[Summary](https://xzhu0027.gitbook.io/blog/big-data-sys/making-sense-of-performance-in-data-analytics-framework)]
* [**Large-scale cluster management at Google with Borg**](https://storage.googleapis.com/pub-tools-public-publication-data/pdf/43438.pdf) **-** Verma et al., EuroSys '15 \[[Summary](https://xzhu0027.gitbook.io/blog/big-data-sys/large-scale-cluster-management-at-google-with-borg)]
* [**Efficient Queue Management for Cluster Scheduling**](https://www.cse.ust.hk/~weiwa/teaching/Fall16-COMP6611B/reading_list/Yaq.pdf) - Rasley et al., EuroSys '16 \[[Summary](https://xzhu0027.gitbook.io/blog/big-data-sys/other-interesting-papers#efficient-queue-management-for-cluster-scheduling-rasley-et-al-eurosys-16)]
* [**Morpheus: Towards Automated SLOs for Enterprise Clusters**](https://www.usenix.org/conference/osdi16/technical-sessions/presentation/jyothi) -Jyothi et al., OSDI '16
* [**Borg, Omega, and Kubernetes**](https://ai.google/research/pubs/pub44843) - Burns et al., ACM Queue '16
* [**Learning Scheduling Algorithms for Data Processing Clusters**](https://web.mit.edu/decima/content/sigcomm-2019.pdf) - Hongzi et al., SIGCOMM '19 

### Scheduling 

* [**Delay Scheduling: A Simple Technique for Achieving Locality and Fairness in Cluster Scheduling**](https://cs.stanford.edu/~matei/papers/2010/eurosys_delay_scheduling.pdf) - Zaharia et al., EuroSys '10 \[[Summary](https://xzhu0027.gitbook.io/blog/big-data-sys/delay-scheduling-a-simple-technique-for-achieving-locality-and-fairness-in-cluster-scheduling)] 
* [**Sparrow: Distributed, Low Latency Scheduling**](https://cs.stanford.edu/~matei/papers/2013/sosp_sparrow.pdf) - Ousterhout et al., SOSP '13 \[[Summary](https://xzhu0027.gitbook.io/blog/big-data-sys/sparrow-distributed-low-latency-scheduling)]
* [**Multi-Resource Packing for Cluster Schedulers**](https://www.cs.cmu.edu/~xia/resources/Documents/grandl_sigcomm14.pdf) - Grandl et al., SIGCOMM '14 \[[Summary](https://xzhu0027.gitbook.io/blog/big-data-sys/multi-resource-packing-for-cluster-schedulers)]
* [**CLARINET: WAN-Aware Optimization for Analytics Queries**](https://www.usenix.org/system/files/conference/osdi16/osdi16-viswanathan.pdf) **-** Viswanathan et al., OSDI '16 \[[Summary](https://xzhu0027.gitbook.io/blog/big-data-sys/clarinet-wan-aware-optimization-for-analytics-queries)]
* [**Monotasks: Architecting for Performance Clarity in Data Analytics Frameworks**](http://kayousterhout.org/publications/sosp17-final183.pdf)  - Ousterhout et al., SOSP '17 \[[Summary](https://xzhu0027.gitbook.io/blog/big-data-sys/monotasks-architecting-for-performance-clarity-in-data-analytics-frameworks)]
* [**Drizzle: Fast and Adaptable Stream Processing at Scale** ](http://shivaram.org/publications/drizzle-sosp17.pdf) **-** Venkataraman et al., SOSP '17 \[[Summary](https://xzhu0027.gitbook.io/blog/big-data-sys/drizzle-fast-and-adaptable-stream-processing-at-scale)]

### Storage

* [**Flat Datacenter Storage**](https://www.usenix.org/system/files/conference/osdi12/osdi12-final-75.pdf) - Nightingale et al., OSDI '12 \[[Summary](https://xzhu0027.gitbook.io/blog/distributed-storage/flat-datacenter-storage)]
* [**Tachyon: Reliable, Memory Speed Storage for Cluster Computing Frameworks**](https://people.csail.mit.edu/matei/papers/2014/socc_tachyon.pdf) - Li et al., SoCC '14 \[[Summary](https://xzhu0027.gitbook.io/blog/distributed-storage/tachyon-reliable-memory-speed-storage-for-cluster-computing-frameworks)]
* [**EC-Cache: Load-balanced, Low-latency Cluster Caching with Online Erasure Coding**](https://www.usenix.org/system/files/conference/osdi16/osdi16-rashmi.pdf) - Rashmi et al., OSDI '16

### Fault Tolerance 

* [**Improving MapReduce Performance in Heterogeneous Environments**](http://courses.cs.vt.edu/cs5204/fall12-kafura/Papers/MapReduce/Map-Reduce-Hadoop.pdf) **-** Zaharia et al., OSDI '08 \[[Summary](https://xzhu0027.gitbook.io/blog/big-data-sys/improving-mapreduce-performance-in-heterogeneous-environments)]
* [**Reining in the Outliers in Map-Reduce Clusters using Mantri** ](https://www.usenix.org/legacy/events/osdi10/tech/full_papers/Ananthanarayanan.pdf)**-** Ananthanarayanan et al., OSDI '10 \[[Summary](https://xzhu0027.gitbook.io/blog/big-data-sys/reining-in-the-outliers-in-map-reduce-clusters-using-mantri)]
* [**Effective Straggler Mitigation: Attack of the Clones** ](https://www.usenix.org/system/files/conference/nsdi13/nsdi13-final231.pdf) **-** Ananthanarayanan et al., NSDI '13 \[[Summary](https://xzhu0027.gitbook.io/blog/big-data-sys/effective-straggler-mitigation-attack-of-the-clones)]
* [**Interruptible Tasks: Treating Memory Pressure AsInterrupts for Highly Scalable Data-Parallel Program**](https://people.cs.uchicago.edu/~shanlu/paper/sosp15-itask.pdf) - Fang et al., SOSP '15 \[[Summary](https://xzhu0027.gitbook.io/blog/big-data-sys/interruptible-tasks-treating-memory-pressure-asinterrupts-for-highly-scalable-data-parallel-progra)]
* [**Lineage Stash: Fault Tolerance Off the Critical Path**](https://dl.acm.org/authorize?N695036) - Wang et al., SOSP '19 \[[Summary](https://xzhu0027.gitbook.io/blog/big-data-sys/other-interesting-papers#lineage-stash-fault-tolerance-off-the-critical-path-wang-et-al-sosp-19)] 

### Misc.

* [**ApproxHadoop: Bringing Approximations to MapReduce Frameworks**](https://dl.acm.org/citation.cfm?id=2694351) - Goiri et al., ASPLOS '15
* [**Yak: A High-Performance Big-Data-Friendly Garbage Collector**](https://www.usenix.org/system/files/conference/osdi16/osdi16-nguyen.pdf) - Nguyen et al., OSDI '16


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://xzhu0027.gitbook.io/blog/cloud-computing/index.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.