Lease Pattern

Lease pattern

In SQL Server, if the primary is no longer available, it is very dangerous to data. Because primary usually is the only entry that user can write bytes into. Primary is not available, so that service is not available to write. To client end, the service still is readable but service is actually dead. In distributed system, high availablity feature try to achieve ZERO downtime. Therefore, effictive detect unavailable resource is required.

Lease pattern uses to controlled resources usage that we should released unused resource peroidly

Windows Server Failover Cluster need to know primary server is no responsed/unavailable, so WSFC performs failover, release dead primary and assign a secondary server as new primary. Lease pattern is introduced to do detection periodicly.

Key concept

Lease pattern is a management tool to manage resource. Regarding to a resource, we have resource provider, and resource user. resource provider is called grantor, and resource user is holder. A lease is negotiated between grantor and holder in a time duration. When lease is not renew in the time interval, lease will timeout and release resource.

implementation

  1. we need to determine what resources to assciated with. In MSSQL, each SQL Server is a resource. HadrRes is a resource type, but an availability group is a HadrRes resource. WSFC is grantor, and SQL Server is a holder.

  2. Lease creation policy. a lease is created by the lease grantor

    1. one user per lease
    2. specify duration
    3. grantor maintain mapping lease and resource
  3. Lease renew Policy

  4. Lease expire Policy

    1. how to release resource from lease when lease timeout
    2. grantor should remove mapping
    3. prepare all cleanup method

example

This is my understanding about WSFC lease pattern usage. when we create an availability group in WSFC.

  1. WSFC register HadrRes type to understand AG interface.
  2. WSFC create availability group resource, called AG01
  3. during bring AG01 online, WSFC prepare and enroll the lease by doing following. HadrRes uses AG01’s property, the primary, to create an event handler. HadrRes binds this event handler with a watchdog callback. WSFC registers the event.
  4. WSFC keeps mapping between the event and primary(resource)
  5. in watchdog callback, WSFC keeps check lease heartbeat. If WSFC doesn’t receive response from heartbeat in a interval, lease timeout. WSFC terminate the lease and release the AG resource.
  6. Based on timeout policy, WSFC should release a primary and desinate a secondary as new Primary. At this point, AG01 resource should offline and bring online. A new lease should generate in this process.

Pro and Con

Pro:

  • Resource management simplicity
  • Efficient Resource Usage. It is time based usage, we can release dead resource in every time interval.
  • Resource Update Simplicity. When you keeps binding between user and resource, user can get updated resource in transparent.
  • Enhance System Reliability. Hey, we know resource is dead in single time interval.

Con:

  • additional Overhead. Each user need a lease to use same resource, but we can use lease pool to limit too much lease being generate in the same time also don’t have to frequently create and release the lease.
  • Additional application logic. well, in windows cluster it is another 10K lines code.
  • Timer watchdog. Not all Operating System has watchdog, but we can use event-based callback to trigger lease check instead.