Search This Blog

Monday, June 10, 2013

MongoDB Unclean Shutdown

My laptop battery ran out of juice and dumped all standby data from RAM.  When I booted the system back up, and tried to start my mongodb instance.  I received the following error:

Unclean shutdown detected.
Please visit for recovery instructions.

If you don't want to read through the documentation to fix the error above re-run mongod with the following usage:

mongod(.exe) --repair --repairpath <(c:)/path/to/data> --dbpath <(c:)/path/to/data>

Once it is complete, start mongo db as usual.

Successful Windows Console Output:
C:\dev\mongodb-win32-i386-2.4.4\bin>mongod --repair --repairpath c:/dev/data --d
bpath c:/dev/data
Mon Jun 10 19:22:16.312
Mon Jun 10 19:22:16.312 warning: 32-bit servers don't have journaling enabled by
 default. Please use --journal if you want durability.
Mon Jun 10 19:22:16.328
Mon Jun 10 19:22:16.359 [initandlisten] MongoDB starting : pid=3936 port=27017 d
bpath=c:/dev/data 32-bit host=samson-pc
Mon Jun 10 19:22:16.359 [initandlisten]
Mon Jun 10 19:22:16.359 [initandlisten] ** NOTE: This is a 32 bit MongoDB binary.
Mon Jun 10 19:22:16.359 [initandlisten] **       32 bit builds are limited to less than 2GB of data (or less with --journal).
Mon Jun 10 19:22:16.359 [initandlisten] **       Note that journaling defaults to off for 32 bit and is currently off.
Mon Jun 10 19:22:16.359 [initandlisten] **       See
Mon Jun 10 19:22:16.359 [initandlisten]
Mon Jun 10 19:22:16.359 [initandlisten] ** NOTE: your operating system version does not support the method that MongoDB
Mon Jun 10 19:22:16.359 [initandlisten] **       uses to detect impending page faults.
Mon Jun 10 19:22:16.359 [initandlisten] **       This may result in slower performance for certain use cases
Mon Jun 10 19:22:16.359 [initandlisten]
Mon Jun 10 19:22:16.359 [initandlisten] db version v2.4.4
Mon Jun 10 19:22:16.375 [initandlisten] git version: 4ec1fb96702c9d4c57b1e06dd34eb73a16e407d2
Mon Jun 10 19:22:16.375 [initandlisten] build info: windows sys.getwindowsversion(major=6, minor=0, build=6002, platform=2, service_pack='Service Pack 2') BOOST_LIB_VERSION=1_49
Mon Jun 10 19:22:16.375 [initandlisten] allocator: system
Mon Jun 10 19:22:16.375 [initandlisten] options: { dbpath: "c:/dev/data", repair: true, repairpath: "c:/dev/data" }
Mon Jun 10 19:22:16.578 [initandlisten] finished checking dbs
Mon Jun 10 19:22:16.578 dbexit:
Mon Jun 10 19:22:16.578 [initandlisten] shutdown: going to close listening sockets...
Mon Jun 10 19:22:16.578 [initandlisten] shutdown: going to flush diaglog...
Mon Jun 10 19:22:16.578 [initandlisten] shutdown: going to close sockets...
Mon Jun 10 19:22:16.578 [initandlisten] shutdown: waiting for fs preallocator...
Mon Jun 10 19:22:16.578 [initandlisten] shutdown: closing all files...
Mon Jun 10 19:22:16.578 [initandlisten] closeAllFiles() finished
Mon Jun 10 19:22:16.578 [initandlisten] shutdown: removing fs lock...
Mon Jun 10 19:22:16.578 dbexit: really exiting now

Wednesday, June 5, 2013

Java and Google Finance

Here are a couple code snippets using JDK6 to fetch data from the Google Finance website.  
Here are some default variables.  The base URL is the google site that will return a json string with requested symbols.  The symbols file is sitting on my Desktop and is just a CSV with 1 column.  Each record is a stock symbol. e.g. JNJ
The Boolean friendlyPrint variable determines the output format.
 private static String defaultBaseUrl = "";  
  private static String defaultSymbolsFile = System.getProperty("user.home")  
   + "/Desktop/NasdaqSymbols.csv";  
  private static boolean friendlyPrint = true;  
This main method first creates an ArrayList of the first 100 stock symbols from the CSV, and then appends them with a comma to the base url.  The URL is submitted and the response is parsed into a Simple-JSON Array. Last, the data is displayed on the console.
  public static void main(String[] args) throws Exception {  
  ArrayList symbols = FileUtils.getTextListValues(defaultSymbolsFile);  
  StringBuilder syms = new StringBuilder();   
  String sb = Util.fetchURL(defaultBaseUrl + syms);  
  JSONArray json = (JSONArray) new JSONParser().parse(sb.substring(3));  
  console.utils.GoogleFinancePrinter.printJsonArray(json, friendlyPrint);  
This is the method called above to fetch the data using the URLConnection object. It reads each line of the response and passes it into a stringbuilder. Then returns the final string.
 static String fetchURL(String url) throws Exception {   
  URL gf = new URL(url);   
  URLConnection yc = gf.openConnection();   
  BufferedReader in = new BufferedReader(new InputStreamReader(   
  StringBuilder sb = new StringBuilder();   
  String inputLine;   
  while ((inputLine = in.readLine()) != null)   
  return sb.toString();   
Here is the method that prints the data on the console. There is more data in the json object than what is displayed in the friendly condition. These are just a few.
  public static void printJsonArray(JSONArray json, boolean friendly) {  
  if (friendly) {  
   for (int i = 0; i < json.size(); i++) {  
   System.out.println("ID: "  
    + ((JSONObject) json.get(i)).get("id"));  
   System.out.println("Symbol: "  
    + ((JSONObject) json.get(i)).get("t"));  
   System.out.println("Name: "  
    + ((JSONObject) json.get(i)).get("name"));  
   System.out.println("Type: "  
    + ((JSONObject) json.get(i)).get("type"));  
   System.out.println("Exchange: "  
    + ((JSONObject) json.get(i)).get("e"));  
   System.out.println("LastTrade: $"  
    + ((JSONObject) json.get(i)).get("l"));  
   System.out.println("Last Trade: "  
    + ((JSONObject) json.get(i)).get("lt"));  
   System.out.println("Change: "  
    + ((JSONObject) json.get(i)).get("c") + "("  
    + ((JSONObject) json.get(i)).get("cp") + "%)");  
  } else {