1、Introduction and Concepts,Advanced RAC Auckland - May 2008,,Session 1, 2008 Julian Dyke,2,Agenda,Introduction Definitions Background OPS versus RAC Reasons for Deployment Alternatives Concepts Discovery,3,Introduction,4,Some RAC Terminology,RAC,LMS,LCK,LMD,FAN,LMON,VIP,ONS,DIAG,OLSNODES,CRS_START,GR
2、D,GCS,GES,TAF,OCFS,OCFS2,OCR,CRS,FCF,SRVCTL,OCRDUMP,VIPCA,GSD,LKDEBUG,CRS_STOP,PI,CRS_STAT,CRSD,OCSSD,CSS,OCRCONFIG,OCRCHECK,CRSCTL,EVMD,OIFCFG,CLUVFY,CRS_REGISTER,CRS_UNREGISTER,5,Definitions What is Single Instance Oracle?,Single instance running on individual server Single database on dedicated s
3、torage DAS - Direct Attached Storage,Local Disk,6,Definitions What is Single Instance Oracle?,Single instance running on individual server Single database on networked storage SAN - Storage Area Network NAS - Network Attached Storage,SAN or NAS,7,Definitions What is RAC?,Multiple instances running o
4、n separate servers (nodes) Single database on shared storage accessible to all nodes Instances exchange information over an interconnect network,( Interconnect ),Shared Storage,Local Disk,Local Disk,Public Network,Private Network,Storage Network,8,Definitions Instances versus Databases,A RAC cluster
5、 includes one database one or more instancesA database is a set of files Located on shared storage Contains all persistent resourcesAn instance is a set of memory structures and processes Contain all temporal resources Normally one instance on each server (node) Can be started and stopped independen
6、tly,9,Background A Brief History of Oracle,10,Background A Brief History of Oracle,Continued,11,Background History of OPS,Oracle Parallel Server (OPS) Oracle 5 OPS developed for DEC/VAX clusters Oracle 6.2 First public release of OPS Oracle 7,0, 7.1, 7.2, 7.3 and 8.0 OPS continued to be developed Or
7、acle 8.1.5 Included Cache Fusion Phase 1 Consistent read blocks transferred across interconnect,12,Background History of RAC,Real Application Clusters (RAC) Oracle 9.0.1 OPS renamed to Real Application Clusters Included Cache Fusion Phase 2 Updated blocks transferred across interconnect Oracle 9.2 i
8、ncluded Oracle Cluster Manager (oracm) for Linux OCFS Version 1 supported on Linux and Windows Oracle 10.1 Introduction of Oracle Cluster Ready Services (CRS) Introduction of Automatic Storage Management (ASM) Oracle 10.2 CRS renamed to Oracle Clusterware OCFS Version 2 supported on Linux,13,OPS ver
9、sus RAC Oracle 8.0.6 and below,Instance 2,Node 2,OPS - Oracle 8.0.6 and below,Interconnect,Current Writes,Consistent Reads,Current Reads,All I/O uses shared storage Enqueues only use interconnect,14,OPS versus RAC Oracle 8.1.5 to Oracle 8.1.7,Instance 2,Node 2,OPS - Oracle 8.1.5 and above - Cache Fu
10、sion Phase 1,Interconnect,Current Writes,Consistent Reads,Current Reads,Current I/O always uses shared storage Consistent reads can use interconnect,15,OPS versus RAC Oracle 9.0.1 and above,Instance 2,Node 2,RAC - Oracle 9.0.1 and above - Cache Fusion Phase 2,Interconnect,Current Writes,Consistent R
11、eads,Current Reads,Current I/O and consistent reads can use interconnect,16,Reasons for Deployment Summary,Users may deploy RAC to achieve:Increasing availabilityIncreasing scalabilityImproving maintainabilityReduction in total cost of ownershipEnsure that you are deploying RAC for one or more of th
12、e above reasons,17,Reasons for Deployment Availability,If one node or instance fails: Database can still be accessed by remaining nodes Sessions on existing nodes are unaffected Sessions on failed node must reconnect Cluster is frozen during part of this process Known as brown-out period,Instance 1,
13、Node 1,Instance 2,Node 2,Interconnect,18,Reasons for Deployment Availability,Deploying RAC: Should achieve less unplanned downtime May require more planned downtime Allows more time to respond to failuresInstance failover means any node can fail without total loss of serviceThere must be overcapacit
14、y in the cluster to survive failover May require: additional hardware additional Oracle and RAC licenses Load can be distributed over all running nodes,19,Reasons for Deployment Availability,Although RAC generally increases availability remember: You can still get data corruptions Human errors / sof
15、tware errors Only one logical copy of data Only one logical copy of application / Oracle softwareThere are still lots of possibilities for human errors Power / network cabling / storage configurationUpgrades and patches are more complex Software can be upgraded on a subset of nodes If the database i
16、s affected then you will still need to arrange some downtime,20,Reasons for Deployment Scalability,Scalability is the relationship between increments of resources and throughput Can be any resource but with RAC normally refers to adding instances Scalability can belinear - optimal but rare non-linea
17、r - suboptimal but normal,Resource,Throughput,Resource,Throughput,Linear,Non-Linear,21,Reasons for Deployment Scalability,RAC overhead means that linear scalability is difficult to achieve Global Cache Services (blocks) Global Enqueue Services (locks)Scaling factor of 1.8 is considered goodDependent
18、 on application design and implementationScaling factor improves with Node affinity Elimination of contention,22,Reasons for Deployment Scalability,Scalability can be improved by: Reducing consumption of resources by applications CPU Disk Network traffic Public network Private network (Interconnect)
19、Reducing contention for resources Rows Blocks Locks Latches I/0,23,Reasons for Deployment Scalability,Oracle provides a number of features which can improve RAC scalability including:Database services Workload balancing Dynamic resource mastering Automatic Segment Space Management (ASSM) Sequences P
20、artitioning Reverse key indexes,24,Reasons for Deployment Manageability,Advantages of RAC Fewer databases to manage Economies of scale Easier to monitor Easier to upgrade Easier to control resource allocation Resources can be shared between applicationsDisadvantages of RAC Upgrades potentially more
21、complex Downtime may affect more applications Requires more experienced operational staff Higher cost / harder to replace,25,Reasons for Deployment Total Cost of Ownership,Reduction in TCO is possible for sites with legacy systems Mainframes / Minicomputers Applications / PackagesRAC option adds 50%
22、 to licence costs except for Users with site licences Standard edition (10.1+, max 4 CPU with ASM)Retrain existing staff or use dedicated staffConsolidation may bring economies of scale Monitoring Backups Disaster Recovery,26,Reasons for Deployment Total Cost of Ownership,Additional resources requir
23、ed Redundant hardware Nodes Network switches SAN fabric Hardware e.g. fibre channel cardsReduction in hardware support costs May not require 24 hour support Viable to hold stock of spare components,27,Other Considerations Application Portability,Most single-instance applications should port to RACAp
24、plication must scale well on single instance This can be difficult to evaluate without testingSome features do not work across instances e.g. e.g. DBMS_ALERT and DBMS_PIPEExternal inputs/outputs may need modification e.g flat files, UTL_FILE directories etcSome RAC features require additional coding
25、 in the application e.g. TAFCode may need upgrading to use RAC functionality e.g. FCF requires JDBC Implicit Connection Cache,28,Other Considerations Alternatives to RAC,Data Guard Physical Standby Introduced in Oracle 7.3 Stable, well proven technology Requires redundant hardware Implemented by man
26、y sites Can be used with RAC Logical Standby Introduced in Oracle 9.2 Still not widely adopted Streams Introduced in Oracle 9.2 Implemented by increasing number of sites Advanced Replication,29,Other Considerations Alternatives to RAC,Symmetric Multiprocessing (SMP) Systems Single Point of Failure S
27、implified configuration May eliminate RAC overheadParallel systems For systems with deterministic input. For example Messaging Data WarehousesOther Clustering Technologies Active / Passive clusters SAN based solutions e.g. SRDF,30,Summary,A successful RAC deployment is dependent on Application desig
28、n and implementation Failover requirements IT infrastructure Flexibility and commitment of IT departmentsBefore deploying RAC, if possible perform proof of concept: Test application Evaluate benefits and costs Learn RAC concepts and administration,31,Concepts,32,RAC Architecture,Instance 1,Node 1,In
29、stance 2,Node 2,Shared Storage,Storage Network,Private Network (Interconnect),Public Network,33,Concepts Shared Storage,Database must be located on shared storage files must be accessible by all instancesDatabase files which must be on shared storage include: Control Files Data Files Online Redo Log
30、s Server Parameter File (if configured)Optional Flashback Recovery Area on shared storage may include: Archived Redo Logs (recommended) Backups Flashback Logs (Oracle 10.1 and above) Change Tracking Writer files (Oracle 10.1 and above),34,Concepts Shared Storage,Shared stored is optional for: Archiv
31、ed redo logs (non-ASM) Backup and recovery files (non-ASM) Executables (Binaries) Password files Parameter files Network configuration files Administrative directories Alert Log Dump Files,35,Concepts Oracle Homes,You can install multiple copies of the Oracle executables on the local disks on each n
32、ode Alternatively you can install a Shared Oracle Home single copy of Oracle executables on shared storageOracle 9.2 Only Oracle database software Oracle 10.1 Cluster Ready Services (CRS) Oracle database software + ASM Oracle 10.2 Oracle Clusterware (CRS) ASM Oracle database software,36,Concepts Red
33、o Threads,In a RAC database there should be one redo log thread per instance Redo log threads can be added using the following syntax:,ALTER DATABASE ADD LOGFILE THREAD 2 GROUP 3 SIZE 51200K, GROUP 4 SIZE 51200K; ALTER DATABASE ENABLE PUBLIC THREAD 2;,Each instance writes to a separate redo log thre
34、ad Redo log files must be on shared storage In the event of a node failure all surviving nodes must be able to access log file of failed instance,37,Concepts Undo Tablespaces,If you are using Automatic Undo Management each instance requires a separate UNDO tablespace Additional undo tablespaces can
35、be added using the following syntax,CREATE UNDO TABLESPACE “UNDOTBS2“ DATAFILE SIZE 25600K AUTOEXTEND ON MAXSIZE UNLIMITED EXTENT MANAGEMENT LOCAL;,38,Concepts Interconnect,Instances communicate with each other over the interconnect (network)Information transferred between instances includes data bl
36、ocks locks SCNsTypically 1Gb Ethernet UDP protocol Often teamed in pairs to avoid SPOFsCan also use Infiniband Fewer levels in stackOther proprietary protocols are available,39,Concepts Internal Structures and Services,Global Resource Directory (GRD) Records current state and owner of each resource
37、Contains queues for waiters, converters and writers Distributed across all instances in clusterGlobal Cache Services (GCS) Implements cache coherency for database Coordinates access to database blocks for instances Maintains GRDGlobal Enqueue Services (GES) Controls access to other resources (locks)
38、 including library cache dictionary cache,40,Concepts Background Processes,Each RAC instance has set of standard background processes e.g. PMON SMON LGWR DBWn ARCn RAC instances use additional background processes to support GCS and GES including LMON LCK0 LMD0 LMSn DIAG,41,Concepts Oracle Clusterwa
39、re,Introduced in Oracle 10.1 (Cluster Ready Services - CRS)Renamed in Oracle 10.2 to Oracle Clusterware Cluster Manager providing Node membership services Global resource management High availability functionsIn Oracle 10.2 includes High Availability framework Allows non-Oracle applications to be ma
40、naged,42,Concepts Oracle Cluster Registry (OCR),The Oracle Cluster Registry (OCR) stores the configuration information for Oracle Clusterware / CRSIntroduced in Oracle 10.1 Replaced Server Management (SRVM) disk/fileSimilar to Windows RegistryLocated on shared storageIn Oracle 10.2 and above the OCR
41、 can be mirrored Maximum two copies,43,Concepts Oracle Cluster Registry (OCR),Defines cluster resources including: Databases Instances RDBMS ASM Database Services Node Applications VIP ONS GSD Listeners,44,Concepts CSS Voting Disk,Known as Quorum Disk / File in Oracle 9iLocated on shared storage acc
42、essible to all instancesUsed to determine RAC instance membershipIn the event of interconnect failure voting disk is used to determine which instance takes control of cluster Avoids split brain In Oracle 10.2 and above can be mirrored Odd number of copies (1, 3, 5 etc),45,Concepts Virtual IP (VIP),N
43、ode application introduced in Oracle 10.1Allows Virtual IP address to be defined for each nodeAll applications connect using Virtual IP addressesIf node fails Virtual IP address is automatically relocated to another nodeOnly applies to newly connecting sessions,46,Concepts Virtual IP,Listener1,Insta
44、nce1,Listener2,Instance2,Listener1,Instance1,Listener2,Instance2,VIP1,After,Before,Node 1,VIP1,VIP2,VIP1,VIP2,Node 1,Node 2,Node 2,47,Concepts Failover,Remaster Enqueue Resources,Remaster Cache Resources,Build Recovery Set,Resource Claim,Roll Forward Recovery Set,Database Partially Available,Node or
45、 Instance Failure,Database Unavailable,Database Fully Available,Time,LMON recovers GRD,SMON recovers Database,Merge failed RedoThreads,48,Concepts Transparent Application Failover (TAF),TAF is Transparent Application FailoverSessions connected to a failed instance will be terminated Uncommitted tran
46、sactions will be rolled backSessions can be reconnected to another instance automatically if using TAF Can optionally re-execute in-progress SELECT statements Statement re-executed with same SCN Fetches resume at point of failure Session state is lost including Session parameters Package variables C
47、lass and ADT instantiations,49,Concepts Fast Application Notification (FAN),Introduced in Oracle 10.1Method by which applications can be informed of changes in cluster status Handle node failures Workload balancingApplications must connect using database servicesCan be notified using Server side cal
48、louts Fast Connection Failover (FCF) ONS API,50,Concepts Database Services,Database Services are logical groups of sessions In Oracle 10.1 and above, each service can have One or more preferred (primary) instances used by default One or more available (secondary) instances used if preferred instance
49、s have failedDatabase Services can be used to support: Workload balancing Transparent Application Failover (TAF) Node affinity Statistics collection Trace collection,51,Concepts Database Services,Listener1,VIP1,Instance1,Listener2,VIP2,Instance2,SERVICE1,SERVICE1,SERVICE1,Listener1,VIP1,Instance1,Listener2,VIP2,Instance2,SERVICE1,SERVICE1,After,Before,52,Concepts Database Services,
