1、Design for Performance,Quality Attributes & Tactics,Performance Evaluation for Various Systems,Performance is generally concerned with how long it takes the system to respond when an event occurs Two typical performance evaluation scenarios A Web-based financial system Events coming from numerous us
2、ers Response measured by transaction per minute An engine control system Events coming from a timer internal to the system Response measured by variation in response time,Server-side Perspective,Characterizing patterns of events arriving and patterns of responses Multiple users or other loading fact
3、ors can be modeled by varying the arrival pattern for events. What matters is the arrival pattern at the server and dependencies within the requests. The response can be characterized by latency, the throughput of the system etc.,Request for Some service,System,Consume some resource,results,Event Ar
4、rival Patterns,Event arrival patterns Periodic E.g., an event arriving every 10 ms in a real-time system Stochastic Events arrive according to some probabilistic distribution Sporadic Irregular event arrival patterns,Response Patterns,Response can be measured by: Latency The time between the arrival
5、 of the stimulus and the systems response to it Throughput E.g., transactions per minute Jitter The variation in latency Events not processed when the system was too busy (miss rate) Data lost when the system was too busy (data loss),A Typical Performance Scenario,“Users initiate 1,000 transactions
6、per minute stochastically under normal operations, and these transactions are processed with an average latency of two seconds.“,Performance General Scenario (1),Source of stimulus One of a number of independent sources, possibly from within system Stimulus: event arrivals with patterns recognized a
7、s Periodic Stochastic Sporadic Artifact The systems services,Performance General Scenario (2),Environment Normal mode Emergency mode Overloaded mode Response Processing the arriving events and generating a response, may cause a change in system environment. Response Measure Latency, deadline, throug
8、hput, jitter, miss rate, data loss,Performance Tactics,The goal of the performance tactics is: To generate a response to an event within some time constraint,Tactics to Control Performance,Events Arrive,Response generated within time constraints,Network,Processing Sequence of an Event,component,A me
9、ssage generated by a component,component,queue,event,transformed,component,queue,response,queue,intermediary result,processed,transformed,Two Contributors to the Response Time (1),When an event arrives it is either processed or blocked for some reason. Response time = resource-consumption-time + blo
10、cked-time Resource consumption Processors, data stores, network bandwidth, memory . System specific resources: Buffers allocated to concurrent processes (producer-consumer) Critical sections (must be executed sequentially),Two Contributors to the Response Time (2),A computation can be blocked becaus
11、e: Contention for resources Single stream of events or multiple streams of events In general more streams more contention greater latency This also depends on the arbitration mechanism for contention,Two Contributors to the Response Time (3),A computation can be blocked because: Availability of reso
12、urces Resource unavailability (e.g., offline, crash, etc.) contributes to latency Dependency on other computation Synchronization Waiting for the results of another computation,Performance Tactics,The two contributors of the response time are both concerned with resources, either consuming it or awa
13、iting it. Consequently all our performance tactics are about resources. Resource demand Resource management Resource arbitration,Resource Demand,Two characteristics of resource demand How often do event streams demand a resource How much of the resource is consumed by each request“Reducing Resources
14、 Required” tactic for reducing latency Increase computational efficiency Improving algorithms Intermediate data: repository VS. regenerationReduce computational overhead E.g., RMI reduces communication requirements Use of intermediaries increases contention; the classic modifiability/performance tra
15、de-off,Example: the Arch. for Tianya BBS,Reducing the Number of Events,Manage event rate E.g. Over-engineered systems reducing the sampling frequency E.g. Limiting the number of users accessing a systemControl frequency of sampling Process events at the rate you can regardless of arrival rate Genera
16、ting a response in a time limit Possible loss of data (events),Controlling the Use of Resources,Bound execution times Place a limit on how much execution time is used to respond to an event. E.g., limit the number of iterations for iterative, data-dependent algorithms.Bound queue sizes Put a limit o
17、n the maximum number of queued events.,Resource Management Tactics (1),Introduce concurrency: process requests in parallel to reduce the blocked time. Multiple threads Load balancing is important hereIncrease available resources Faster processors More processors Additional memories Faster network Co
18、st/performance quality attributes trade-off,Resource Management Tactics (2),Maintain multiple copies of either data or computation More buffers, more processes Client-server: reduce contention that would occur if all computation takes place on a central server Caching (memory hierarchy) : same data
19、replicated on different repositories Data synchronization and consistency is an important issue here P2P downloading,From : HPC, Kai Wang, CS DEPT, USD,21,A Simple Example,Add 1 to 101+2+3+4+5+6+7+8+9+10Use a single CPU computerfor (i = 1; i = 10; i+)sum = sum + i;Suppose each operation needs 1 seco
20、nd9 seconds,22,A Simple Example,Use two processorsSolution one1+2+3+4+5 (processor 1)6+7+8+9+10 (processor 2)Solution two1+3+5+7+9 (processor 1)2+4+6+8+10 (processor 2),From : HPC, Kai Wang, CS DEPT, USD,23,A Simple Example,Computation Each processor computesIndependently and simultaneouslyProcessor
21、 1 computes 1+3+5+7+9Processor 2 computes 2+4+6+8+10CommunicationProcessor 2 sends value sum to processor 1Processor 1 receives the value sum as sum2ComputationProcessor 1 computes 5 operations + Message passing (=5 seconds)Save 4 seconds,for (i = pid; i = 10; i = i+2)sum = sum + i ;,sum=sum+sum2,Fr
22、om : HPC, Kai Wang, CS DEPT, USD,Linkedin 2003,LinkedIn 2006,Scheduling,Resources must be scheduled in case of contention. Scheduling algorithms Priority assignment Dispatching Scheduling criteria Optimal resource usage, request importance, minimizing latency, maximizing throughput, preventing starv
23、ation to ensure fairness Preemption Anytime Only at special moment Not preempting executing processes,First-in/First-out,FIFO treats all requests for resources as equals and satisfy them in turn. Request priority is not considered. A request might be stuck for a long time.,Fixed-priority Scheduling,
24、A fixed priority is assigned to each request and that assignment will remain unchanged. High-priority requests are better served Low-priority requests may wait for an arbitrarily long timeCommon prioritization strategies: Semantic importance Deadline monotonic higher priority to streams with shorter
25、 deadlines Rate monotonic - higher priority to streams with shorter periods,Dynamic-priority Scheduling,The priority of the event requests are calculated during the execution of the system. Round robin Event requests are refreshed after the requests are served one term. Earliest deadline first Refre
26、sh priority according to the remain deadline of the events.,Static Scheduling,Pre-emption points and the sequence of assignment to the resource are determined offline.,Performance Tactics Hierarchy,Performance,Increase Computational EfficiencyReduce Computational OverheadManage Event RateControl Fre
27、quency of Sampling,Scheduling policy,Introduce ConcurrencyMaintain Multiple CopiesIncrease Available Resources,Resource Demand,Resource Management,Resource Arbitration,Events Arrive,Responses Generated within Time Constraints,Appendix: an example,SUNS HIGH PERFORMANCE COMPUTING REFERENCE ARCHITECTURE,
