Random Notes
  • Introduction
  • Reading list
  • Theory
    • Index
      • Impossibility of Distributed Consensus with One Faulty Process
      • Time, Clocks, and the Ordering of Events in a Distributed System
      • Using Reasoning About Knowledge to analyze Distributed Systems
      • CAP Twelve Years Later: How the “Rules” Have Changed
      • A Note on Distributed Computing
  • Operating System
    • Index
  • Storage
    • Index
      • Tachyon: Reliable, Memory Speed Storage for Cluster Computing Frameworks
      • Exploiting Commutativity For Practical Fast Replication
      • Don’t Settle for Eventual: Scalable Causal Consistency for Wide-Area Storage with COPS
      • Building Consistent Transactions with Inconsistent Replication
      • Managing Update Conflicts in Bayou, a Weakly Connected Replicated Storage System
      • Spanner: Google's Globally-Distributed Database
      • Bigtable: A Distributed Storage System for Structured Data
      • The Google File System
      • Dynamo: Amazon’s Highly Available Key-value Store
      • Chord: A Scalable Peer-to-peer Lookup Service for Internet Applications
      • Replicated Data Consistency Explained Through Baseball
      • Session Guarantees for Weakly Consistent Replicated Data
      • Flat Datacenter Storage
      • Small Cache, Big Effect: Provable Load Balancing forRandomly Partitioned Cluster Services
      • DistCache: provable load balancing for large-scale storage systems with distributed caching
      • Short Summaries
  • Coordination
    • Index
      • Logical Physical Clocks and Consistent Snapshots in Globally Distributed Databases
      • Paxos made simple
      • ZooKeeper: Wait-free coordination for Internet-scale systems
      • Just Say NO to Paxos Overhead: Replacing Consensus with Network Ordering
      • Keeping CALM: When Distributed Consistency is Easy
      • In Search of an Understandable Consensus Algorithm
      • A comprehensive study of Convergent and Commutative Replicated Data Types
  • Fault Tolerance
    • Index
      • The Mystery Machine: End-to-end Performance Analysis of Large-scale Internet Services
      • Gray Failure: The Achilles’ Heel of Cloud-Scale Systems
      • Capturing and Enhancing In Situ System Observability for Failure Detection
      • Check before You Change: Preventing Correlated Failures in Service Updates
      • Efficient Scalable Thread-Safety-Violation Detection
      • REPT: Reverse Debugging of Failures in Deployed Software
      • Redundancy Does Not Imply Fault Tolerance
      • Fixed It For You:Protocol Repair Using Lineage Graphs
      • The Good, the Bad, and the Differences: Better Network Diagnostics with Differential Provenance
      • Lineage-driven Fault Injection
      • Short Summaries
  • Cloud Computing
    • Index
      • Improving MapReduce Performance in Heterogeneous Environments
      • CLARINET: WAN-Aware Optimization for Analytics Queries
      • MapReduce: Simplified Data Processing on Large Clusters
      • Dryad: Distributed Data-Parallel Programs from Sequential Building Blocks
      • Resource Management
      • Apache Hadoop YARN: Yet Another Resource Negotiator
      • Mesos: A Platform for Fine-Grained Resource Sharing in the Data Center
      • Dominant Resource Fairness: Fair Allocation of Multiple Resource Types
      • Large-scale cluster management at Google with Borg
      • MapReduce Online
      • Delay Scheduling: A Simple Technique for Achieving Locality and Fairness in Cluster Scheduling
      • Reining in the Outliers in Map-Reduce Clusters using Mantri
      • Effective Straggler Mitigation: Attack of the Clones
      • Resilient Distributed Datasets: A Fault-Tolerant Abstraction for In-Memory Cluster Computing
      • Discretized Streams: Fault-Tolerant Streaming Computation at Scale
      • Sparrow: Distributed, Low Latency Scheduling
      • Making Sense of Performance in Data Analytics Framework
      • Monotasks: Architecting for Performance Clarity in Data Analytics Frameworks
      • Drizzle: Fast and Adaptable Stream Processing at Scale
      • Naiad: A Timely Dataflow System
      • The Dataflow Model:A Practical Approach to Balancing Correctness, Latency, and Cost in Massive-Scale
      • Interruptible Tasks:Treating Memory Pressure AsInterrupts for Highly Scalable Data-Parallel Program
      • PACMan: Coordinated Memory Caching for Parallel Jobs
      • Multi-Resource Packing for Cluster Schedulers
      • Other interesting papers
  • Systems for ML
    • Index
      • A Berkeley View of Systems Challenges for AI
      • Tiresias: A GPU Cluster Managerfor Distributed Deep Learning
      • Gandiva: Introspective Cluster Scheduling for Deep Learning
      • Workshop papers
      • Hidden Technical Debt in Machine Learning Systems
      • Inference Systems
      • Parameter Servers and AllReduce
      • Federated Learning at Scale - Part I
      • Federated Learning at Scale - Part II
      • Learning From Non-IID data
      • Ray: A Distributed Framework for Emerging AI Applications
      • PipeDream: Generalized Pipeline Parallelism for DNN Training
      • DeepXplore: Automated Whitebox Testingof Deep Learning Systems
      • Distributed Machine Learning Misc.
  • ML for Systems
    • Index
      • Short Summaries
  • Machine Learning
    • Index
      • Deep Learning with Differential Privacy
      • Accelerating Deep Learning via Importance Sampling
      • A Few Useful Things to Know About Machine Learning
  • Video Analytics
    • Index
      • Scaling Video Analytics on Constrained Edge Nodes
      • Focus: Querying Large Video Datasets with Low Latency and Low Cost
      • NoScope: Optimizing Neural Network Queriesover Video at Scale
      • Live Video Analytics at Scale with Approximation and Delay-Tolerance
      • Chameleon: Scalable Adaptation of Video Analytics
      • End-to-end Learning of Action Detection from Frame Glimpses in Videos
      • Short Summaries
  • Networking
    • Index
      • Salsify: Low-Latency Network Video through Tighter Integration between a Video Codec and a Transport
      • Learning in situ: a randomized experiment in video streaming
      • Short Summaries
  • Serverless
    • Index
      • Serverless Computing: One Step Forward, Two Steps Back
      • Encoding, Fast and Slow: Low-Latency Video Processing Using Thousands of Tiny Threads
      • SAND: Towards High-Performance Serverless Computing
      • Pocket: Elastic Ephemeral Storage for Serverless Analytics
      • Fault-tolerant and Transactional Stateful Serverless Workflows
  • Resource Disaggregation
    • Index
  • Edge Computing
    • Index
  • Security/Privacy
    • Index
      • Differential Privacy
      • Honeycrisp: Large-Scale Differentially Private Aggregation Without a Trusted Core
      • Short Summaries
  • Misc.
    • Index
      • Rate Limiting
      • Load Balancing
      • Consistency Models in Distributed System
      • Managing Complexity
      • System Design
      • Deep Dive into the Spark Scheduler
      • The Actor Model
      • Python Global Interpreter Lock
      • About Research and PhD
Powered by GitBook
On this page
  • What is the actor model?
  • Address
  • Handling failure
  • Transparent life-cycle management
  • When to use actors?
  • When not to use actors?
  • Drawbacks:
  • Next steps:

Was this helpful?

  1. Misc.
  2. Index

The Actor Model

PreviousDeep Dive into the Spark SchedulerNextPython Global Interpreter Lock

Last updated 5 years ago

Was this helpful?

This post is primarily a summarization of a given by John Murray. You may not get anything out of it that you wouldn’t get out of the talk, but I think it could be helpful when you want to refer to a particular concept. In addition, for my own benefit, writing about them helps solidify it in my head.

What is the actor model?

The actor model is a conceptual model to deal with concurrent computation. It defines some general rules for how the system’s components should behave and interact with each other.

Properties of actor:

  • Unlike thread, which no longer exists when it finishes, actors are persistent.

  • Unlike goroutine/threads/future, which are stateless, actors encapsulate internal state.

  • Actors are asynchronous

What can actors do?

  • Create new actors - similar to the main thread creates other threads.

  • Receive messages and in response:

    • make local decisions(e.g. alter local state)

    • perform arbitrary, side-effecting action(e.g. writing to a database, a log file or anything that change the global state of your application.)

    • send messages to other actors

    • respond to the sender zero or more times. Whereas, in procedural programming paradigm, if you call a function, the function either returns nothing or returns exactly one thing back.

  • Process exactly one message at a time. (The messages are stored in actors' mailboxes until they're processed.)

Actors do not communicate by sharing memory; instead, share memory by communicating.

Example - Checking account:

Suppose if the current balance of a shared account between Alice and Bob is $80 and we want to make sure that the balance is non-negative at any time. Alice wants to withdraw $60 and Bob wants to withdraw $50.

Approach #1:

struct Checking {
    balance int
}

if( Checking.balance > withDrawAmt) {
    Checking.balance -= withdrawAmt;
    return true;
} else {
    return false;
}

However, it's easy to create a situation that violates the invariants - If Alice and Bob submit their request at the same time, it's likely that the interleaving execution will lead to negative balance.

Approach #2 - Using locks:

struct Checking {
    balance int
    lock Mutex
}

Checking.lock.Lock();
success = false; 
if( Checking.balance > withdrawAmt) {
    Checking.balance -= withdrawAmt;
    success = true;
} else {
    success = false;
}
Checking.lock.Free();
return success;

Introducing mutex will solve the problem for us : only one person will be updating at any time. While it is correct, in real world, using locks are often expensive and complicated. You need to make sure that the order is correct and there can be deadlock situations.

In general, while locks seem to be the natural remedy to uphold the invariants with multiple threads, in practice they are inefficient and easily lead to deadlocks in any application of real-world scale. Even worse, distributed locks, while exists, offer limited potential for scaling out.

Approach #3 - Using actors:

Actor Checking {
    var balance = 80
    
    // Looking for incoming messages 
    def receive = {
        case Withdraw(amt) =>
            if( balance > amt) {
                balance -= amt
                // Instead of return, we send a message to the sender
                sender sendMsg true
            } else {
                sender sendMsg false
            }
        }
}

If we make two requests:

// send message to withdraw 60 dollars
Checking sendMsg Withdraw(50)

// send message to withdraw 80 dollars
Checking sendMsg Withdraw(50)

Two requests are stored in the mailbox, and the rule of only processing one message at a time will make sure that only one request goes through and the other person will receive a "false" message. Modifying the internal state of the checking actor is only possible via messages, which are processed one at a time eliminating races when trying to keep invariants. Even better, the senders are not blocked as they do when using locks - Millions of actors can be efficiently scheduled on a dozen of threads reaching the full potential of modern CPUs

Properties of communication:

  • No channels or intermediates(e.g. CSP)

  • "Best effort" delivery(i.e. no matter what your underlying protocol is, on the actor's perspective, there are no time-outs/retries)

  • At-most-once delivery

  • Message can take arbitrary long to be delivered. In actor model, there is no concept of 'time')

  • No message ordering guarantees

Address

Address identifies an actor. However, it may also represent a proxy/forwarder to an actor(e.g. a load balancer). The addresses contain location(e.g. IP addresses) and transportation information(e.g. TCP/UDP). They give us the notion of location transparency. As a programmer, we don't need to care about where the actor lives, as long as we can send message to it. In other words, we can have actors in the same process, or different machines, but how you communicate are completely the same across all of them.

Handling failure

The running state of an actor is monitored and managed by another actor, which is called the supervisor.

Properties of supervision:

  • Constantly monitors running state of an actor

  • Performs actions based on the state of the actor(e.g. restart the actor)

Transparent life-cycle management

  • Address do not change during restart

  • Mailboxes(Queues) are persistent outside the actor instance

When to use actors?

  1. Processing pipeline

  2. Streaming data

  3. multi-user concurrency

  4. System with high up-time requirement

  5. Applications with shared state

When not to use actors?

  1. Non-concurrent systems

  2. Performance critical applications

Note: It's important to keep in mind that, actors are just an abstraction. They exist on top of processes and threads. If you need fine-control over the running threads(e.g. interrupts), they you want to use something else.

Drawbacks:

  • "Too much actors"

  • Testing

  • Debugging

Next steps:

But, who supervise the supervisor? In the actor model, there will be a supervision tree. Similar to a organizational structure within a company, the managers manage their direct reports and they are managed by their managers. At the top, there will be a oracle provided by the framework(e.g. the in akka). It never dies and has some default behaviors about how to handle exceptions and errors.

- Bernstein et al., 2017

- Boyd-Wickizer et al., 2012

- Wu et al., 2018

talk
root guardian
Don't use Actors for concurrency
Indexing in an Actor-Oriented Database
Non-scalable locks are dangerous
Anna: A KVS For Any Scale
One address may represent many actors(pool)
One actor may have many addresses(A, B, C uses different addresses)
Addresses "encapsulate" the mailbox and the actor