Scheduler Activations

        

1. Questions
2. Notes from Reviews:
1. Worst paper yet?
2. Don’t evaluate how often common case / uncommon case is
3. Why not just control threads from userlevell
4. Are kernel threads inherently more expensive?
5. How integrate with mem. mgmt?

                                              i.     Not – too expensive for a thread

6. Can malicious program use critical section trick to commandeer processors?
7. Done on a not very up-to-date system?

                                              i.     Lag in TOCS publishing – takes 2 years

3.  
4. Context:
1. Late 80s: cheap shared-memory multiprocessors were appearing
2. Threads were seen as the way to build parallel programs

                                              i.     Allow work on multiple processors independently

                                             ii.     Coordinate for synchronization

3. 2 approaches:

                                              i.     user threads: purely userlevel code for:

1.   context switching

2.   synchronization

3.   scheduling

                                             ii.     kernel threads: same features in the kernel (e.g. mach threads, windows NT threads)

4. Kernel Rules:

                                              i.     Must completely control scheduling of processors

1.   Can’t let user code have control

2.   Can let user code advice

3.   Can’t let advice be used for correctness – otherwise commandeering

                                             ii.     Kernel multiplexes processors between processes – can’t be done within a process

5. Thread operations:

                                              i.     spawn a thread

                                             ii.     synchronize (e.g. locking)

                                           iii.     terminate a thread

                                           iv.     schedule a new thread

6. Issue:

                                              i.     User level threads fast; no need to go to kernel

1.   Expensive to enter kernel

2.   Kernel approach must be general & work for all applications

                                             ii.     But: Kernel events pre-empt user threads

1.   Block high-priority threads on I/o, or preemption for time slicing

2.   Schedule kernel threads on separate mechanism; may run wrong thread or too many threads

a.    e.g. wake up low-priority thread instead of high-priority

3.   Correctness: kernel may schedule wrong threads – ones that are blocked waiting for a user thread blocked in kernel: may not have enough kernel threads to schedule a new user thread and make progress.

5. Goals:
1. No kernel intervention in common case
2. No processor idles when any program has threads to schedule
3. No high-priority thread waits for processor while low-priority thread executes
4. When a thread blocks in kernel, processor can be rescheduled with a different thread
5. User-level portion can be customized
6. Key observations:
1. Kernel needs information/notification from user level about:

                                              i.     When threads runnable

                                             ii.     When processors not needed

2. User level threads need information from kernel:

                                              i.     When thread pre-empted

                                             ii.     When thread resumed

                                           iii.     When processor removed

                                           iv.     When processor returned

3. Goal: don’t want to spend lots of time communicating information to kernel; is too expensive
6. Solution:
1. HIGH LEVEL SOLUTION: interface / abstraction

                                              i.     Expose info to across information

2. Rules:

                                              i.     Kernel controls which kernel threads run, how many run

                                             ii.     User controls which user threads run

                                           iii.     Kernel notifies user of any scheduling events (e.g. taking away a kernel thread, adding a kernel thread, blocking & unblocking)

                                           iv.     User notifies kernel when needs fewer or more processors

3. User code runs in “scheduler activations”, not threads

                                              i.     Scheduler activations run to completion; they are never re-scheduled

                                             ii.     System upcalls into user-level scheduler on all interesting events

1.   Thread blocked

2.   Thread wakes up

3.   Thread pre-empted

4.   Processor available

5.   Processor removed

                                           iii.     Up-call may pre-empt existing thread, causes notification that two threads are runnable

                                           iv.     KEY POINT:

1.   No need to get permission to pre-empt; just notify on another thread afterward

2.   Detail: on page fault, may delay of fault again on same point

3.   Detail: may wait until next time a kernel thread is available if no threads currently running

                                             v.     Application notifies kernel of available parallelism

1.   Fewer contexts needed

2.   More contexts could be used

3.   NOTE: no need to notify of other events, such as which thread is running now or context switches

                                           vi.     SHOW EXAMPLE

                                          vii.     Critical sections

1.   Problem: may preempt thread holding ready list lock. When send up activation, can’t get lock to schedule itself -- DEADLOCK

2.   Preemption control kernel lets user decide which threads should not be preempted

a.    May require pinning memory to avoid page faults

b.   Yields control to user level from kernel

c.    To avoid deadlock, must be a guarantee – not just a wish

3.   Recovery:

a.    When preempt thread holding a lock, can continue it instead of running scheduler. On release of lock, goes back to upcall and into scheduler

b.   Mechanism:

                                                                                                    i.     Goal: zero overhead for common case

                                                                                                   ii.     Solution: mark critical sections in assembly

                                                                                                 iii.     Copy code to new place

                                                                                                 iv.     New code returns control to scheduler on releasing a lock instead of continuing

                                                                                                   v.     Problem: locks acquired in one indirect function call, released in another

                                                                                                 vi.     Key idea: use knowledge of source code for fast common case behavior – zero overhead.

                                                                                                vii.     Used in Linux for trap handling: certain places are marked as “safe” for traps, stores a fixup routine to recover. E.g. copy from user

                                                                                              viii.      

                                        viii.     Performance:

1.   QUESTION: What do you want to show?

a.    A: no cost for cpu-bound operations

b.   A: Better than both for blocking operations

2.   As fast as fast threads when CPU bound

3.   Fixed amount better than kernel threads when I/O bound – no unnecessary blocking

                                           ix.     QUESTION: where did we see this before?

1.   Spin, Exokernel – allow choice of what thread to run next when CPU given to a process

8. Key points: Knowledge + control
1. Kernel notifies User-level of what it is doing
2. user level can control what runs next when kernel provides a processor
3. Kernel retains control of proecessor
9. Issues/comments
1. Re-implemented in BSD
2. Idea adopted in solaris but:

                                              i.     Took reservation idea for critical sections, nothing else

3. User level threads largely died – for server applications, access to I/O, system calls important