Search This Blog

Monday, July 29, 2013

Change Linux system clock safely with bash script

I learned the hard way a month or two back that you cannot just change the server time when running zookeeper with distributed cloud services on a single server.  Often changing the system clock to a time which is greater than the zookeeper session time (2x the tick time, ~ 4000 ms by default) you can easily break zookeeper session synchronization.

The fix is to change the time little by little until it is where you want it.  Here is an example in which I speed up the clock by 1 second every 10 seconds.  This has been tested and worked while cloud services are running without breaking anything.

#! /usr/bin/env bash

#Increase server time by 960 seconds(16mins) over a 9600 second(2hrs40mins) period.

for i in {1..960}
do
later=$(date --date="+1 second")
echo $later
date -s "$later"
sleep 10
done