LimitTheMaximumConcurrency

# Limit the Maximum Concurrency

Define a Semaphore
LimitTheMaximumConcurrency_1DefineASemaphore
Define a Conflict Graph
LimitTheMaximumConcurrency_1DefineAConflictGraph
Use a Semaphore across Different Tasks
LimitTheMaximumConcurrency_1UseASemaphoreAcrossDifferentTasks

This chapters discusses how to limit the maximum concurrency or parallelism of workers running inside tasks.

# Define a Semaphore

Taskflow provides a mechanism, tf::Semaphore, for you to limit the maximum concurrency in a section of tasks. You can let a task acquire/release one or multiple semaphores before/after executing its work. A task can acquire and release a semaphore, or just acquire or just release it. A tf::Semaphore object starts with an initial count. As long as that count is above 0, tasks can acquire the semaphore and do their work. If the count is 0 or less, a task trying to acquire the semaphore will not run but goes to a waiting list of that semaphore. When the semaphore is released by another task, it reschedules all tasks on that waiting list.

    tf::Executorexecutor(8);//createanexecutorof8workers
    tf::Taskflowtaskflow;
    tf::Semaphoresemaphore(1);//createasemaphorewithinitialcount1

    intcounter=0;

    //create1000independenttasksinthetaskflow
    for(size_ti=0;i<1000;i++){
    taskflow.emplace([&](tf::Runtime&rt){
    rt.acquire(semaphore);
    counter++;//onlyoneworkerwillincrementthecounteratanytime
    rt.release(semaphore);
    });
    }

    executor.run(taskflow).wait();

The above example creates 1000 tasks with no dependencies between them. Each task increments `counter` by one. Since this increment operation is protected by the semaphore initialized with a count of `1`, no multiple workers will run this operation at the same time. In other words, the semaphore limits the parallelism of the 1000 tasks to 1.

It is your responsibility to ensure the semaphore stays alive during the execution of tasks that acquire and release it. The executor and taskflow do not manage lifetime of any semaphores.

We can create a semaphore with a different count value, such as 3, to limit the parallelism of independent tasks to 3.

    tf::Executorexecutor(8);//createanexecutorof8workers
    tf::Taskflowtaskflow;

    tf::Semaphoresemaphore(3);//createasemaphorewithinitialcount3

    //createataskthatacquiresandreleasesthesemaphore
    taskflow.emplace([&](tf::Runtime&rt){
    rt.acquire(semaphore);
    std::cout<<"A"<<std::endl;
    rt.release(semaphore);
    });

    //createataskthatacquiresandreleasesthesemaphore
    taskflow.emplace([&](tf::Runtime&rt){
    rt.acquire(semaphore);
    std::cout<<"B"<<std::endl;
    rt.release(semaphore);
    });

    //createataskthatacquiresandreleasesthesemaphore
    taskflow.emplace([&](tf::Runtime&rt){
    rt.acquire(semaphore);
    std::cout<<"C"<<std::endl;
    rt.release(semaphore);
    });

    //createataskthatacquiresandreleasesthesemaphore
    taskflow.emplace([&](tf::Runtime&rt){
    rt.acquire(semaphore);
    std::cout<<"D"<<std::endl;
    rt.release(semaphore);
    });

    //createataskthatacquiresandreleasesthesemaphore
    taskflow.emplace([&](tf::Runtime&rt){
    rt.acquire(semaphore);
    std::cout<<"E"<<std::endl;
    rt.release(semaphore);
    });

    executor.run(taskflow).wait();

    #Onepossibleoutput:A,B,andCrunconcurrently,DandErunconcurrently
    ABC
    ED

tf::Semaphore is also useful for limiting the maximum concurrency across multiple task groups. For instance, you can have one task acquire a semaphore and have another task release that semaphore to impose concurrency on different task groups. The following example serializes the execution of five pairs of tasks using a semaphore rather than explicit dependencies.

    tf::Executorexecutor(4);//createsanexecutorof4workers
    tf::Taskflowtaskflow;
    tf::Semaphoresemaphore(1);

    intN=5;
    intcounter=0;//non-atomicintegercounter

    for(inti=0;i<N;i++){
    tf::Taskf=taskflow.emplace([&](tf::Runtime&rt){
    rt.acquire(semaphore);
    counter++;
    }).name("from-"s+std::to_string(i));

    tf::Taskt=taskflow.emplace([&](tf::Runtime&rt){
    counter++;
    rt.release(semaphore);
    }).name("to-"s+std::to_string(i));

    f.precede(t);
    }

    executor.run(taskflow).wait();

    assert(counter==2*N);

Without semaphores, each pair of tasks, e.g., `from-0 -> to-0`, will run independently and concurrently. However, the program forces each `from` task to acquire the semaphore before running its work and not to release it until its paired `to` task is done. This constraint forces each pair of tasks to run sequentially, while the order of which pair runs first is up to the scheduler.

# Define a Conflict Graph

One important application of tf::Semaphore is conflict-aware scheduling using a conflict graph. A conflict graph is a undirected graph where each vertex represents a task and each edge represents a conflict between a pair of tasks. When a task conflicts with another task, they cannot run together. Consider the conflict graph below, task `A` conflicts with task `B` and task `C` (and vice versa), meaning that `A` cannot run together with `B` and `C` whereas `B` and `C` can run together.

We can create one semaphore of one concurrency for each edge in the conflict graph and let the two tasks of that edge acquire the semaphore. This organization forces the two tasks to not run concurrently.

    tf::Executorexecutor;
    tf::Taskflowtaskflow;

    tf::Semaphoreconflict_AB(1);
    tf::Semaphoreconflict_AC(1);

    //taskAcannotruninparallelwithtaskBandtaskC
    tf::TaskA=taskflow.emplace([&](tf::Runtime&rt){
    rt.acquire(conflict_AB,conflict_AC);
    std::cout<<"A"<<std::endl;
    rt.release(conflict_AB,conflict_AC);
    });

    //taskBcannotruninparallelwithtaskA
    tf::TaskB=taskflow.emplace([&](tf::Runtime&rt){
    rt.acquire(conflict_AB);
    std::cout<<"B"<<std::endl;
    rt.release(conflict_AB);
    });

    //taskCcannotruninparallelwithtaskA
    tf::TaskC=taskflow.emplace([&](tf::Runtime&rt){
    rt.acquire(conflict_AC);
    std::cout<<"C"<<std::endl;
    rt.release(conflict_AC);
    });

    executor.run(taskflow).wait();

    #Onepossibleoutput:BandCrunconcurrentlyafterA
    A
    BC

tf::Runtime::acquire can acquire multiple semaphores at a time, similarly for tf::Runtime::release which can release multiple semaphores at a time. When acquiring a semaphore, the calling worker will corun until the semaphore is successfully acquired. This corun behavior allows us to avoid any deadlock that could possibly happen when using semaphores with other tasks.

# Use a Semaphore across Different Tasks

You can use tf::Semaphore across different types of tasks, such as async tasks, taskflow tasks, and your application code. tf::Semaphore does not impose any restriction on which task to use.

    tf::Executorexecutor;
    tf::Taskflowtaskflow1,taskflow2;

    intcounter(0);
    size_tN=2000;

    for(size_ti=0;i<N;i++){
    //acquireandreleasethesemaphorefromataskintaskflow1
    taskflow1.emplace([&](tf::Runtime&rt){
    rt.acquire(s);
    counter++;
    rt.release(s);
    });

    //acquireandreleasethesemaphorefromataskinanothertaskflow2
    taskflow2.emplace([&](tf::Runtime&rt){
    rt.acquire(s);
    counter++;
    rt.release(s);
    });

    //acquireandreleasethesemaphorefromanasynctask
    executor.async([&](tf::Runtime&rt){
    rt.acquire(s);
    counter++;
    rt.release(s);
    });
    }

    executor.wait_for_all();
    assert(counter==3*N);