tag:blogger.com,1999:blog-61674700395240682572024-03-13T08:12:45.784-07:00Champion of CyrodiilA blog dedicated to the random encounters of computer and technology related issues.Champion Of Cyrodiilhttp://www.blogger.com/profile/06792737627887393792noreply@blogger.comBlogger77125tag:blogger.com,1999:blog-6167470039524068257.post-62268245950399905812020-06-18T23:32:00.013-07:002021-02-18T06:35:05.519-08:00Linux, Newest Kernel, Old Hardware, Windows 10<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-DA8di6QLZ-M/XuxbbpUbNoI/AAAAAAAAH1Y/qcROrpEzIw4M_tLln5Gbb0FKSLBHGchlACLcBGAsYHQ/s1600/Screenshot%2Bfrom%2B2020-06-19%2B02-24-59.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="900" data-original-width="1600" height="360" src="https://1.bp.blogspot.com/-DA8di6QLZ-M/XuxbbpUbNoI/AAAAAAAAH1Y/qcROrpEzIw4M_tLln5Gbb0FKSLBHGchlACLcBGAsYHQ/s640/Screenshot%2Bfrom%2B2020-06-19%2B02-24-59.png" width="640" /></a></div>
<div style="text-align: center;">
<br /></div>
I just had to do this post! August will be 4 years of great performance for this Dell Precision 7710. Kudos to Dell for great hardware design. I did an in depth review here in August 2016, <a href="http://championofcyrodiil.blogspot.com/2016/08/linux-work-and-vpn-requirements.html">Linux, Newest Kernel, Newest Hardware, Windows 10</a> when I first set up the system.<div><br /></div><div>First, big Shout Out to one of my best friends and co-workers that introduced me to Debian, Nate! Thanks for all the tips and support over the years. Even though you couldn't keep up w/ the bleeding edge that is... testing! LoL<br />
<br />
Next, I must say the folks at Oracle have done great at keeping VirtualBox Free to use at home, and stable enough to keep upgrading the same VM for four years. Also, great work with the new VBoxSVGA graphics controller with 3D acceleration. It not only works great, but screen resize and responsiveness with graphics intensive applications is flawless, even on this older Quadro M!<br />
<br />
Microsoft, I'm shocked. I thought for sure I would be wiping this VM at some point. Never did I think it would be running my entire Windows Experience flawlessly virtualized with Office, VPN and Updates for years. I have almost 4 years of content packed in there! I even use the USB passthrough with Web Cam and Smart Card reader, to support the use of Skype!<br />
<br />
Last but not least, Linux Debian community. Your packaging and persistence to compatible releases and regression tests has helped keep us Debian users stable. At this point I'm running on Buster, and have made significant upgrades.<br />
<br />
I can't recommend this configuration enough. Only down side has been the Broadcom 5580 Smart Card reader, which has been broken in Virtualbox for at least 10 years. USB card readers work just fine.<div><br /></div><div>I should also say I spent a few years doing photogrammetry development. That means this machine's CPU has been pegged at 100% for >24 hours continuously, while enormous amounts of disk have been consumed and deleted, with thousands of small to large (4GB+) files. It is an absolute work horse.</div><div><br /></div><div>I've also carried this thing around the U.S. and dragged it through airports and scanners on multiple occasions. The disk is still encrypted with LVM, and I've updated the password more times than I can remember at this point.</div><div><br /></div><div>Pleased to say I am now smoothly running with:<br />
<br />
VirtualBox 6.1.10<br />
Windows 10 Pro Build 18362.19h1_release.190318-1202<br />Debian 4.19.118-2+deb10u1 (2020-06-07) x86_64 GNU/Linux<div><br /></div><div>The only question now is, when do I get a new PC? I still have 32 GB of RAM, 8 Logical CPUs running on a 6th Gen Core i7. The GPU is totally adequate for almost all workloads, and the SSD SMART logs show no issue with the hard drive. Wifi is fine, as well as sound. All pluggable ports work well and the Keyboard and mouse pad has no issues or stuck keys. The backlighting for the keyboard still works as well.</div><div><br /></div><div>One thing I did not cover in the original post (because I must have not seen the error) is the microcode for the Intel Chipset. Make sure to install that if you see the TSC_Deadline Failure during boot. This was a "WARNING" i saw for years and ignored. It may have effected performance but I didn't notice. Nevertheless, I aim to understand all errors on my Linux PC, and solve them.</div><div><br /></div><div>From <a href="https://unix.stackexchange.com/questions/410854/tsc-deadline-disabled-due-to-errata" target="_blank">Stack Exchange</a>:</div><div><br /></div><div><div>TSC deadline support is nice to have, but not vital. The kernel has an elaborate framework for timekeeping and timed event handling; TSC deadline is one possible implementation of event handling, but not the only one. On CPUs which support it, it is nice to have though, because it’s very efficient.</div><div><br /></div><div>To upgrade your microcode and hopefully re-enable TSC deadline support, you can install the microcode update packages from Debian’s contrib and non-free repositories. To do so, edit your /etc/apt/sources.list to ensure that your Debian repository definitions include main, contrib and non-free; then run</div><div><br /></div><div><font face="courier"># sudo apt update</font></div><div><br /></div><div>followed by</div><div><br /></div><div><font face="courier"># sudo apt install intel-microcode</font></div><div><br /></div><div>(for Intel CPUs) or</div><div><br /></div><div><font face="courier"># sudo apt install amd64-microcode</font></div><div><br /></div><div>(for AMD CPUs). Once that’s done, reboot, and your microcode should be updated. If TSC deadline support is re-enabled, you won’t see the error message at boot, and you’ll see tsc_deadline_timer in the flags lines of /proc/cpuinfo.</div><div><br /></div><div>The Debian wiki has more information on <a href="https://wiki.debian.org/Microcode" target="_blank">microcode updates</a>.</div></div></div></div>Champion Of Cyrodiilhttp://www.blogger.com/profile/06792737627887393792noreply@blogger.com0tag:blogger.com,1999:blog-6167470039524068257.post-78443862521877307812020-02-08T22:19:00.001-08:002020-02-08T22:22:41.050-08:00Synology Follow-UpIn the previous post (<a href="https://championofcyrodiil.blogspot.com/2020/02/synology-home-nas.html">https://championofcyrodiil.blogspot.com/2020/02/synology-home-nas.html</a>) I configured the Synology DS218j through a virtual network using an Ubuntu VM as the DHCP Server. Today I wanted to back up some of my files, so I needed to configure some kind of share.<br />
<h2>
Enable SSHd on the VM</h2>
<div>
<ol>
<li>$ sudo apt-get install -y openssh-server</li>
<li>$ sudo systemctl enable ssh.service</li>
<li>$ sudo systemctl start ssh.service</li>
</ol>
</div>
<h2>
Create an Account on Synology</h2>
<div>
You will need an account to connect to on the Synology. You could use the admin account, but don't. This was easy enough and walking through the Wizard after clicking "Users" somewhere in Control Panel, I created an account named 'charlie'. </div>
<div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://1.bp.blogspot.com/-C6A-e59BUzE/Xj-gyC0KmoI/AAAAAAAAHi4/QFEHnVcq4aUsA_L-gUlByrV-2lU7FVc1wCLcBGAsYHQ/s1600/NewUser-Synology.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1147" data-original-width="1577" height="290" src="https://1.bp.blogspot.com/-C6A-e59BUzE/Xj-gyC0KmoI/AAAAAAAAHi4/QFEHnVcq4aUsA_L-gUlByrV-2lU7FVc1wCLcBGAsYHQ/s400/NewUser-Synology.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><i>The Control Panel is easy to find when you log in with the Web Browser on port 5000 over HTTP<br />HTTP is Plain-Text and thus Not Secure. This is another good reason we are working on a private virtual network, as there are no other devices that could be snooping.</i></td></tr>
</tbody></table>
<h2>
Check SMB Settings</h2>
</div>
<div>
By default SMB is enabled, but I discovered I needed to enable SMB version 3.0 on the Synology! You'll find this under the "Advanced" SMB settings. </div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-BczpRP3rVHM/Xj-hynHIE3I/AAAAAAAAHjI/vZnrk0IshcU37MFAWeS2dkYBw1yAdfieQCLcBGAsYHQ/s1600/SMB-Settings-Synology.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1191" data-original-width="1600" height="297" src="https://1.bp.blogspot.com/-BczpRP3rVHM/Xj-hynHIE3I/AAAAAAAAHjI/vZnrk0IshcU37MFAWeS2dkYBw1yAdfieQCLcBGAsYHQ/s400/SMB-Settings-Synology.png" width="400" /></a></div>
<div style="text-align: center;">
<br /></div>
<h2>
Create a "New Folder"</h2>
<div>
Creating an account by itself does not create a Share/Folder that can be accessed from a remote client. You'll need to create a new folder as well. Again, the UI is intuitive and you should be able to navigate to "Folders" in the control panel, and create a new one, giving your user account created in the previous step "Customized" permissions. I enabled "Full Control". </div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-KEDwp2IzU2o/Xj-hXbhqieI/AAAAAAAAHjA/dh5muOtnq6kOCsxKE9ypz7ZwY3XdrAxVgCLcBGAsYHQ/s1600/NewFolder-Synology.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1101" data-original-width="1565" height="281" src="https://1.bp.blogspot.com/-KEDwp2IzU2o/Xj-hXbhqieI/AAAAAAAAHjA/dh5muOtnq6kOCsxKE9ypz7ZwY3XdrAxVgCLcBGAsYHQ/s400/NewFolder-Synology.png" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div>
Also, when creating the folder I set a password and enabled disk Encryption. Filename lengths are restricted to ~170 characters in your storage folder if encryption is enabled.</div>
<h2>
Mount the Synology Share using CIFS</h2>
<div>
You'll need to install 'cifs-utils' using apt-get install first. Then you can easily add the entry to the /etc/fstab using sudo.</div>
<div>
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://1.bp.blogspot.com/-_OrOqzVaaxE/Xj-h-flBLzI/AAAAAAAAHjM/ODEC0eMFK2wXQgT1psVFP4mR7ss0z0rBACLcBGAsYHQ/s1600/CIFS-Mount-Synology.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="629" data-original-width="1165" height="215" src="https://1.bp.blogspot.com/-_OrOqzVaaxE/Xj-h-flBLzI/AAAAAAAAHjM/ODEC0eMFK2wXQgT1psVFP4mR7ss0z0rBACLcBGAsYHQ/s400/CIFS-Mount-Synology.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><i>Note the 'user' option. This allows you to mount with the 'vubuntu' user, making it read/write approved for the SSH User. Using `sudo mount` works too, but then accessing the VM remotely would require an SSH as root. You can only SSH as root after modifying the "PermitRootLogin" setting in /etc/ssh/sshd_config.</i></td></tr>
</tbody></table>
<h2 style="text-align: left;">
Was all that Worth it???</h2>
<div>
I would have to say it was. The goal was to back up my Desktop PC's data to the Synology, and I've achieved peek throughput of 540 Mbps while using WinSCP to move files from the Desktop's E:\ Storage drive to the Synology over the Cat5 cable. Which is 67.5MB/sec.</div>
<div>
<br /></div>
<div>
While the connection is 1Gbps, Secure Shell and Virtualization are at play. Again, this isolates the Synology from the web, while still providing high throughput. Don't forget to run your Virus Scan on files before backing them up!</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-ixJwCtZmyt8/Xj-jR-N_OgI/AAAAAAAAHjc/-XSO91DWimUmsrAIDyOcdBQsa0_L66dWwCLcBGAsYHQ/s1600/Peek%2BWrite%2BSpeed%2BSynology.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1038" data-original-width="1346" height="492" src="https://1.bp.blogspot.com/-ixJwCtZmyt8/Xj-jR-N_OgI/AAAAAAAAHjc/-XSO91DWimUmsrAIDyOcdBQsa0_L66dWwCLcBGAsYHQ/s640/Peek%2BWrite%2BSpeed%2BSynology.png" width="640" /></a></div>
<div>
<br /></div>
Champion Of Cyrodiilhttp://www.blogger.com/profile/06792737627887393792noreply@blogger.com0tag:blogger.com,1999:blog-6167470039524068257.post-85002083199722808332020-02-08T11:22:00.002-08:002020-02-08T22:22:26.807-08:00Synology Home NAS<h2 style="clear: both; text-align: center;">
Virtual Networked NAS</h2>
<h2>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://1.bp.blogspot.com/-g5nYg2enRzg/Xj8QYtwysOI/AAAAAAAAHic/pMh4J7AJFfg4TtGlZEEC8MBtyYIsNbhIACLcBGAsYHQ/s1600/Untitled%2BDiagram%2B%25281%2529.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1144" data-original-width="1305" height="560" src="https://1.bp.blogspot.com/-g5nYg2enRzg/Xj8QYtwysOI/AAAAAAAAHic/pMh4J7AJFfg4TtGlZEEC8MBtyYIsNbhIACLcBGAsYHQ/s640/Untitled%2BDiagram%2B%25281%2529.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><i style="font-size: small;">Everything Configured from My Desktop PC running Windows 10; </i><i style="font-size: small; text-align: start;">Color-Coded based on Trust, where Green is obviously the most trusted.</i></td></tr>
</tbody></table>
<div style="text-align: center;">
<br /></div>
<b>Purchase and Put Together:</b></h2>
<ul>
<li>Synology 2 bay NAS DiskStation DS218j (Diskless) - $175.84</li>
<ul>
<li><a href="https://www.amazon.com/Synology-bay-DiskStation-DS218j-Diskless/dp/B076G6YKWZ">https://www.amazon.com/Synology-bay-DiskStation-DS218j-Diskless/dp/B076G6YKWZ</a></li>
</ul>
<li>Seagate IronWolf 4TB NAS Internal Hard Drive (x2) - $210.58</li>
<ul>
<li><a href="https://www.amazon.com/dp/B07H289S79">https://www.amazon.com/dp/B07H289S79</a></li>
</ul>
</ul>
<div>
Total: <u>$386.42</u></div>
<div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://1.bp.blogspot.com/-VQr9x42wvBw/Xj8Cd33jqoI/AAAAAAAAHgY/7tbt6JTpKcM-LkkebZpcHikFTc6nGN4XACLcBGAsYHQ/s1600/Synology.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="900" data-original-width="1600" height="225" src="https://1.bp.blogspot.com/-VQr9x42wvBw/Xj8Cd33jqoI/AAAAAAAAHgY/7tbt6JTpKcM-LkkebZpcHikFTc6nGN4XACLcBGAsYHQ/s400/Synology.jpg" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><i>Both Seagate IronWolf 4TB Drives Installed in Synology DS218j</i></td></tr>
</tbody></table>
</div>
<div>
<ul>
</ul>
<h2>
Software Configuration:</h2>
<br />
<ul>
<li>Install Hyper-V on Windows 10: <a href="https://docs.microsoft.com/en-us/virtualization/hyper-v-on-windows/quick-start/enable-hyper-v">https://docs.microsoft.com/en-us/virtualization/hyper-v-on-windows/quick-start/enable-hyper-v</a></li>
<li>
Download Ubuntu ISO: <a href="http://releases.ubuntu.com/18.04/ubuntu-18.04.3-desktop-amd64.iso">http://releases.ubuntu.com/18.04/ubuntu-18.04.3-desktop-amd64.iso</a></li>
</ul>
</div>
<div>
<h3>
Create Virtual Switch for Desktop NIC:</h3>
</div>
<div>
<ol>
<li>From Hyper-V Manager, Click "Virtual Switch Manager" on the Left-Hand "Action" Panel</li>
<li>Select "External" for Create Virtual Switch Type</li>
<li>Set Name as value that will be intuitive for your Desktop NIC. e.g. 'Wired External Switch"</li>
<li>Choose the Ethernet Device Connected Directly to Synology NAS Device</li>
</ol>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://1.bp.blogspot.com/-T8gg4bk7xcQ/Xj7_W-LkMXI/AAAAAAAAHfs/u-DlnjS1uQsEdzbqWni-8DVXVg_SbMCawCEwYBhgL/s1600/HyperV-ExternalSwitch.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1028" data-original-width="1080" height="304" src="https://1.bp.blogspot.com/-T8gg4bk7xcQ/Xj7_W-LkMXI/AAAAAAAAHfs/u-DlnjS1uQsEdzbqWni-8DVXVg_SbMCawCEwYBhgL/s320/HyperV-ExternalSwitch.png" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><i>Creating an External Switch in Hyper-V<br />Think of this as a 'physical' switch between your CPU and Network Card, with only 1 port (Your NIC!)</i></td></tr>
</tbody></table>
<h3>
Create Virtual Machine and Install using ISO</h3>
</div>
<div>
<a href="https://docs.microsoft.com/en-us/virtualization/hyper-v-on-windows/quick-start/quick-create-virtual-machine">https://docs.microsoft.com/en-us/virtualization/hyper-v-on-windows/quick-start/quick-create-virtual-machine</a></div>
<div>
<br /></div>
<div>
Don't forget to add a second virtual NIC to the VM and attach it to the previously created external switch.</div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://1.bp.blogspot.com/-Bga9lSqT4K8/Xj8D-d2XKJI/AAAAAAAAHgk/Bi3hP8D5sq8ubT4RNlye_bxo9dF7WC7egCLcBGAsYHQ/s1600/HyperV-VM-Config.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1030" data-original-width="1082" height="380" src="https://1.bp.blogspot.com/-Bga9lSqT4K8/Xj8D-d2XKJI/AAAAAAAAHgk/Bi3hP8D5sq8ubT4RNlye_bxo9dF7WC7egCLcBGAsYHQ/s400/HyperV-VM-Config.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><i>Creating "Dual-Homed" VM w/ 2 Virtual NICs<br />One for Synology, One for Internet(NAT)</i></td></tr>
</tbody></table>
<div style="text-align: center;">
<br /></div>
<h3>
Static IP for "Wired External Switch"</h3>
<div>
You've now got everything connected. When you boot ubuntu you'll see 1 NIC has an IP (dhcp) from Hyper-V and provides Internet Connectivity. The second is not enabled at all. You'll need to enable this by editing <span style="font-family: "courier new" , "courier" , monospace;">/etc/network/interfaces</span> as follows:</div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://1.bp.blogspot.com/-ZRA1PlAT_V0/Xj8E8qVpvGI/AAAAAAAAHg8/NFqZHVtIgAo-_fFxRfrg8ylGdq7HGRG8wCLcBGAsYHQ/s1600/Ubuntu-StaticIP-Config.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="595" data-original-width="1070" height="221" src="https://1.bp.blogspot.com/-ZRA1PlAT_V0/Xj8E8qVpvGI/AAAAAAAAHg8/NFqZHVtIgAo-_fFxRfrg8ylGdq7HGRG8wCLcBGAsYHQ/s400/Ubuntu-StaticIP-Config.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><i>Configure Ubuntu to have a Static-IP. This is normal for Servers that provide DHCP<br />Ensure 'eth0' or 'eth1' corresponds with the External Switch chosen above.</i></td></tr>
</tbody></table>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-Lgr0DWMlBc0/Xj8GdX6szEI/AAAAAAAAHhc/d7ODhkcNmusWIlhCY9gwZTgvMfH81XErACLcBGAsYHQ/s1600/Ubuntu-StaticIP-Config-P2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="940" data-original-width="1151" height="326" src="https://1.bp.blogspot.com/-Lgr0DWMlBc0/Xj8GdX6szEI/AAAAAAAAHhc/d7ODhkcNmusWIlhCY9gwZTgvMfH81XErACLcBGAsYHQ/s400/Ubuntu-StaticIP-Config-P2.png" width="400" /></a></div>
<div style="text-align: center;">
<i>'sudo ifup eth0' will bring your NIC online after editing the interfaces file</i></div>
<h3 style="text-align: left;">
DHCP Configuration (This part can be tricky)</h3>
<div>
Follow the instructions here: <a href="https://help.ubuntu.com/community/isc-dhcp-server">https://help.ubuntu.com/community/isc-dhcp-server</a></div>
<div>
I've used the 192.168.10.0/32 network in this example to match up with the information provided in the link above. Make sure to check your syntax after editing the file. `ps -ef | grep dhcpd` should show the server is running. If there are errors, use `journalctl` to view any errors, fix them, and use `sudo systemctl restart isc-dhcp-server.service`. Once DHCP is running and everything is connected, you should see a Lease for the Synology using `cat /var/lib/dhcp/dhcpd.leases'.</div>
<div>
<br />
Make sure to enable 'authoritative' DHCP, since this DHCP server is the only one on this External Switch.</div>
<div>
<br /></div>
<h3>
Set up your Synology, Download the OS with VM</h3>
<div>
When the Synology gets an IP, it should get the first IP in your DHCP range. For me this was 192.168.10.10. Inside your VM, open a browser and go to http://192.168.10.10 </div>
<div>
You will be redirected to set up the device. Your device wont have an internet connection because we did not configure the VM to act as a gateway/router. So you'll need to click the link provided in the Synology setup, and Download the OS software to the VMs Downloads folder. Once complete, verify the checksum `md5sum ~/Downloads/DSM_DS218j_24922.pat`. This ensure you are getting the right software and no corruption occured during download. Then switch back to the Synology configuration Tab, and Choose "Select a File/Browse", navigate to the Downloads folder, and 'upload' the .PAT file to the Synology via the web interface. Finish the configuration and make sure to remember your NAS does NOT HAVE INTERNET CONNECTIVITY. This is so that you can ensure it is secure and not downloading software you do not explicitly provide.</div>
<div>
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://1.bp.blogspot.com/-grIIE5HAecM/Xj8JR_YNqXI/AAAAAAAAHh4/d94vgA1alsUKjWs1r4sTqm1RAFTCHJ6nwCLcBGAsYHQ/s1600/Ubuntu-StaticIP-Config-P2.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1300" data-original-width="1600" height="323" src="https://1.bp.blogspot.com/-grIIE5HAecM/Xj8JR_YNqXI/AAAAAAAAHh4/d94vgA1alsUKjWs1r4sTqm1RAFTCHJ6nwCLcBGAsYHQ/s400/Ubuntu-StaticIP-Config-P2.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><span style="font-style: italic;">Once configured using Defaults, you should have 2 drives providing redundant storage with 3.6TB of Storage</span></td></tr>
</tbody></table>
<h2 style="text-align: left;">
What Next?</h2>
<div>
Now that everything is connected you'll want to think about how you want to access your synology from other devices. You could simply connect the Synology to your Wifi Router now and access it via wifi. However, this would expose it to the internet, so make sure you want to do that.</div>
<div>
<br /></div>
<div>
Another option would be to configure the Virtual Machine as a "Reverse-Proxy" for HTTP/HTTPS or set up the routing tables to forward specific ports and provide NATing back to the requester. There are several possibilities at this point and I hope you have found this tutorial useful and informative.<br />
<br />
Read more on how I configure SMB w/ SCP between the Host OS, VM and Synology devices in the next post, <a href="https://championofcyrodiil.blogspot.com/2020/02/synology-follow-up.html">https://championofcyrodiil.blogspot.com/2020/02/synology-follow-up.html</a></div>
Champion Of Cyrodiilhttp://www.blogger.com/profile/06792737627887393792noreply@blogger.com0tag:blogger.com,1999:blog-6167470039524068257.post-31356876818709057022019-06-15T13:38:00.000-07:002019-06-16T04:45:50.853-07:002005 Ford Expedition A/C Compressor Bypass BeltHere are the signs your A/C is failing and you're in for some serious repairs.<br />
<ol>
<li>You smell a burning while A/C is running. Not like burning rubber, more like burning electronics or wires/metal.</li>
<li>You immediately turn off the vehicle because of the smell, and then later when you turn it on, NOTHING happens. No turn over, no rough start, just a click and silence.</li>
<li>Your Battery is fine, the starter is fine, and MAYBE it starts up again a while later.</li>
</ol>
<div>
In my case, this was the A/C failing. The compressor is depicted below as part #9. It is part of the serpentine belt pulley configuration. #8 is the crankshaft pulley connected to the engine, and as you can see, WILL NOT TURN OVER IF ANY OF THE OTHER PULLEYS ARE SEIZED. This is because the Starter (which starts the engine by turning over the crankshaft) doesnt have enough 'juice' to force the frozen pulley.</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-YkuC9jkDOdQ/XQVXpHnnbgI/AAAAAAAAHF8/iWwbVzUGhB0a3EnIHNayAUk-OlVjex6WQCLcBGAs/s1600/2005-ford-expedition-engine-diagram-2003-ford-expedition-serpentine-belt-routing-and-timing-belt-diagrams-300x210.gif" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1151" data-original-width="1600" height="460" src="https://1.bp.blogspot.com/-YkuC9jkDOdQ/XQVXpHnnbgI/AAAAAAAAHF8/iWwbVzUGhB0a3EnIHNayAUk-OlVjex6WQCLcBGAs/s640/2005-ford-expedition-engine-diagram-2003-ford-expedition-serpentine-belt-routing-and-timing-belt-diagrams-300x210.gif" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div>
<br /></div>
<br />
In my case, #9 froze up so tight the belt started squealing, burning up, and snapped in half. These belts typically only handle a little over 260°F before they break. With #9 seized up, and a 5.4L V8 forcing the belt to slide around it, eventually it will heat up and break. At that point all of the check engine lights come on your dashboard, and you will lose power generation (battery wont charge, #3), power steering (hard to turn the wheel, #5), Cooling for the Engine, (overheating, #7) which will result in the need to replace the engine if it gets too hot (>190-210°), and of course A/C (#9).<br />
<br />
So, now you have no belt, and #9 is seized. You could have it towed, and spend $1500+ dollars getting the A/C components replaced, flushed, and re-charged. In my case, I had pulled forward in to my driveway and wanted to do the work myself, but needed to move the vehicle. So I started looking in to 'bypassing' #9!<br />
<br />
This would look something like this.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-Y6CXoh_LY0I/XQVXuazTJcI/AAAAAAAAHGA/mc8-Kc2X7NEiKms8Gec43L_A-QSjUmtXACLcBGAs/s1600/2005-ford-expedition-engine-diagram-2003-ford-expedition-serpentine-belt-routing-and-timing-belt-diagrams-300x210-bypass-ac.gif" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1151" data-original-width="1600" height="460" src="https://1.bp.blogspot.com/-Y6CXoh_LY0I/XQVXuazTJcI/AAAAAAAAHGA/mc8-Kc2X7NEiKms8Gec43L_A-QSjUmtXACLcBGAs/s640/2005-ford-expedition-engine-diagram-2003-ford-expedition-serpentine-belt-routing-and-timing-belt-diagrams-300x210-bypass-ac.gif" width="640" /></a></div>
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
If you're paying attention, you might notice the most obvious issue here is figuring out how long the new belt needs to be. Another thing I chose to do was replace pulley #2. Not only are the bearings likely old on this pulley, but in my case the pulley is 'smooth' and not 'grooved'. I was concerned about the belt 'slipping' off the pulley, so i replaced it with an <a href="https://www.autozone.com/external-engine/idler-pulley">aftermarket pulley</a>.<br />
<br />
To figure out how long of a belt you need, use a piece of string and run it through the same path. Measure the string and purchase a belt of the same length.<br />
<br />
On my 2005, 5.4L V8 Expedition the options are:<br />
<ul>
<li>Duralast 840K6 - 84.06" (Bypassing the A/C Clutch Pulley)</li>
<li>Duralast 1015K6 - 101.5" (Stock Belt, including the A/C Clutch Pulley)</li>
</ul>
(Duralast is just what is at my local autozone. Just make sure to compare size, rib count, etc.)<br />
You may notice on websites like Autozone, you cannot search by 'length', only by part number. The pattern for these duralast belts seem to be 1/10" increments, and I guess K6 has to do with the 6 "ribs" on the inside of the belt.<br />
<br />
e.g. 101.5 Inches is 1015K6 and 84.0 inches is 840K6, and so on.<br />
<br />
<a href="https://www.autozone.com/cooling-heating-and-climate-control/belt">Link to Autozone's Site with Belts</a> ~ $25<br />
<a href="https://www.autozone.com/external-engine/idler-pulley">Link to Autozone's Site with Idler Pulleys</a> ~ $20<br />
Breaker Bar ~ $17<br />
<b>Total Cost: $62</b><br />
<br />
Make sure you have a breaker bar (1/2" drive) that plugs in to the square hole in #1. Pushing downward (away from the belt), you should feel the spring loaded tensioner move.<br />
<br />
To replace the belt,<br />
<ol>
<li>Start with dropping a loop straight down under #8 looking down at the engine standing in front of the car. YOU DON'T NEED TO GO OVER YOUR FAN BLADES, just around it!</li>
<li>Keeping your left hand at the 9 o'clock on #8 so the belt doesn't fall off, with your right hand, guide the belt clockwise around #7, continue over and under #6, still keeping your left hand at the 9 o'clock position on #8. (Read that step a few times)</li>
<li>Continue around #5, skip #4 and over #3.</li>
<li>you can probably move your left hand now! Slip it over #2 and around the tensioner.</li>
<li>Using the breaker bar, adjust the tensior and go back to #4 (which is a smooth pulley) and slip it over the pulley. Release the tensior.</li>
<li>Double check EVERYTHING ensuring the belt is in the grooves, and around all pulleys. You'll likely notice if there is an issue.</li>
</ol>
<div>
This may feel impossible when you start, but stick with it and you'll find it to be pretty easy when you are done.</div>
<br />
<br />
It will look like this after replacing the Idler Pulley (#2) and using the shorter belt:<br />
<img alt="AC Bypass 1.jpg" height="480" src="https://www.explorerforum.com/forums/index.php?attachments/ac-bypass-1-jpg.98564/" width="640" /><br />
<br />
<img alt="AC Bypass 3.jpg" height="480" src="https://www.explorerforum.com/forums/index.php?attachments/ac-bypass-3-jpg.98566/" width="640" /><br />
<br />
You'll notice the belt goes from #2 (the new black pulley in the top left), around the tensioner to the crank pulley #8. Aka "Bypassing #9".<br />
<br />
You should now be able to drive the car, however will not have A/C.<br />
<br />
<b>NOTE:</b> I am not a certified mechanic nor am I a mechanical engineer. Here are a few other things to consider.<br />
<ul>
<li>A shorter belt with different sized pulleys and less of them will create a modified load on the engine. The tensioner provides a 'factory' tension, so I think it should be fine, but it is not how the car was designed so there could be additional issues with this that I am not aware of.</li>
<li>The 'modified' load on the crankshaft is probably marginal compared to the load of say, towing a trailer with cement bags, but perhaps still worth consideration. Comments welcome if you're a mechanical engineer and have thoughts on this.</li>
<li>I'm too tired to go back outside and take a photo, so credit to <a href="https://www.explorerforum.com/forums/index.php?threads/successful-02-4-6l-ac-bypass.465281/">this forum post</a> for great photos of a 4.6L V8, which is VERY similar to mine.</li>
<li>Be ready to make multiple trips to the store. I got a Duralast 800K6 (79.?") the first time and it was too short. No free lunch here, just measure the best you can using string/twine and hope you got it right.</li>
<li>If your A/C is fine now, but car is like mine and old, consider getting a belt and breaker bar for your trunk. This can be done in a parking lot and could save you a tow for a long trip.</li>
<li>There are likely to be typos, quirks in grammar and terminology, or instances above where I mention pulley X and meant pulley Y. This isn't rocket science so just use your sense and I think you will figure it out.</li>
</ul>
<div>
<br /></div>
<div>
I wrote this blog because I couldn't find a great resource for the 2005 Ford Expedition 5.4L V8 Triton Engine. Later I came across the forum post in the notes above, and finally got motivated to get that second belt (840K6). I hope this is helpful for someone else.<br />
<br />
6/16/19 Update: After letting the Battery trickle charge overnight (it's been sitting for months), I fired it up this morning and it has been running for more than 25 minutes. Took it for a spin down the street and all seems well for now. Brakes are rusty, Power steering is a little tight, but hopefully all that smooths out with some additional use.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-NVvaN_EPhgE/XQYrdHwpYKI/AAAAAAAAHGQ/tlkoK_-noZsKQ9SIwNHRZ3U0-dsTu18eACLcBGAs/s1600/AC%2BBypass.gif" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="656" data-original-width="369" src="https://1.bp.blogspot.com/-NVvaN_EPhgE/XQYrdHwpYKI/AAAAAAAAHGQ/tlkoK_-noZsKQ9SIwNHRZ3U0-dsTu18eACLcBGAs/s1600/AC%2BBypass.gif" /></a></div>
<br /></div>
Champion Of Cyrodiilhttp://www.blogger.com/profile/06792737627887393792noreply@blogger.com18tag:blogger.com,1999:blog-6167470039524068257.post-88651074055221923902019-03-24T10:01:00.001-07:002019-03-24T10:02:22.053-07:00Unity3D with QGIS and GIMPIt has been a long time since I have posted anything on here. I now have my own commercial drone (and FAA license) with a nice camera, and I've integrated several open source projects so that I can create 3D models from imagery/video (photogrammetry).<br />
<br />
So now I am looking to create a sort of unity experience so that I can showcase places I've visited in 3D, and embed them with the larger general area. I think I will have a good amount of learning to do before I'm able to create transitions from 'high altitude' scenes to 'low altitude' scenes where vegetation might appear. I don't think I want to try to "Place Mass Trees" with this large of an area.<br />
<br />
I am calling the concept "ODT" or "Open Digital Twin" (Copyright OTT LLC 2019).<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://3.bp.blogspot.com/-_hkPqBlqQT8/XJe2i0nJcKI/AAAAAAAAG-o/yJvi0IKfOI8AocmTqillE6X0vyOeJWgpQCLcBGAs/s1600/Unity-ODT.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1080" data-original-width="1600" height="430" src="https://3.bp.blogspot.com/-_hkPqBlqQT8/XJe2i0nJcKI/AAAAAAAAG-o/yJvi0IKfOI8AocmTqillE6X0vyOeJWgpQCLcBGAs/s640/Unity-ODT.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
As you can see, the result is actually pretty nice. You can obtain stereo DEMs (digital elevation models) from the USGS using WMS service calls. I then created a Normal Map, and also used USGS WMS to fetch a high altitude imagery color map.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
When creating the height map for unity, you'll need to ensure you are saving the map as "Raw Image File" with Planar mapping.</div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<a href="https://4.bp.blogspot.com/-ao2ltuX56O0/XJe3EA6bgAI/AAAAAAAAG-w/nYM84AIW3n8qUnkDcQJJOr3eRQiqFaV5QCLcBGAs/s1600/Unity-ODT.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1080" data-original-width="1600" height="428" src="https://4.bp.blogspot.com/-ao2ltuX56O0/XJe3EA6bgAI/AAAAAAAAG-w/nYM84AIW3n8qUnkDcQJJOr3eRQiqFaV5QCLcBGAs/s640/Unity-ODT.png" width="640" /></a>There is still plenty of work to be done, but I thought this might be a fun update to get me back in the blogging mode. It's been too many years, and I've come a long way from the original work I did with Java & Hadoop. Hopefully these GIS and 3D skills will couple nicely with my enterprise experience to bring 3D to the enterprise!</div>
Champion Of Cyrodiilhttp://www.blogger.com/profile/06792737627887393792noreply@blogger.com0tag:blogger.com,1999:blog-6167470039524068257.post-46590019941407348982018-05-21T09:16:00.000-07:002018-05-21T09:16:07.931-07:00Repairing Corrupted LUKS Encrypted FilesystemThis morning I noticed that my USB backup drive would not mount properly. The first thing I did was check the dmesg log from Debian/Linux...<br />
<br />
<blockquote class="tr_bq">
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: xx-small;">[1962217.824829] usb 1-5.1: New USB device found, idVendor=0bc2, idProduct=3300</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: xx-small;">[1962217.824832] usb 1-5.1: New USB device strings: Mfr=1, Product=2, SerialNumber=3</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: xx-small;">[1962217.824834] usb 1-5.1: Product: Desktop </span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: xx-small;">[1962217.824835] usb 1-5.1: Manufacturer: Seagate </span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: xx-small;">[1962217.824837] usb 1-5.1: SerialNumber: 2GHNRSVE </span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: xx-small;">[1962217.825589] usb-storage 1-5.1:1.0: USB Mass Storage device detected</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: xx-small;">[1962217.825927] scsi host4: usb-storage 1-5.1:1.0</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: xx-small;">[1962218.832372] scsi 4:0:0:0: Direct-Access Seagate Desktop 0130 PQ: 0 ANSI: 4</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: xx-small;">[1962218.833253] sd 4:0:0:0: Attached scsi generic sg1 type 0</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: xx-small;">[1962218.833370] sd 4:0:0:0: [sdb] 3907029168 512-byte logical blocks: (2.00 TB/1.82 TiB)</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: xx-small;">[1962218.833691] sd 4:0:0:0: [sdb] Write Protect is off</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: xx-small;">[1962218.833694] sd 4:0:0:0: [sdb] Mode Sense: 2f 08 00 00</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: xx-small;">[1962218.833991] sd 4:0:0:0: [sdb] No Caching mode page found</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: xx-small;">[1962218.833996] sd 4:0:0:0: [sdb] Assuming drive cache: write through</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: xx-small;">[1962218.842262] sdb: sdb1</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: xx-small;">[1962218.843668] sd 4:0:0:0: [sdb] Attached SCSI disk</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: xx-small;">[1962219.029068] sd 4:0:0:0: [sdb] tag#0 FAILED Result: hostbyte=DID_ERROR driverbyte=DRIVER_SENSE</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: xx-small;">[1962219.029072] sd 4:0:0:0: [sdb] tag#0 Sense Key : Hardware Error [current] [descriptor] </span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: xx-small;">[1962219.029074] sd 4:0:0:0: [sdb] tag#0 Add. Sense: No additional sense information</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: xx-small;">[1962219.029077] sd 4:0:0:0: [sdb] tag#0 CDB: ATA command pass through(16) 85 06 20 00 00 00 00 00 00 00 00 00 00 00 e5 00</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: xx-small;">[1962219.111836] sd 4:0:0:0: [sdb] tag#0 FAILED Result: hostbyte=DID_ERROR driverbyte=DRIVER_SENSE</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: xx-small;">[1962219.111839] sd 4:0:0:0: [sdb] tag#0 Sense Key : Hardware Error [current] [descriptor] </span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: xx-small;">[1962219.111841] sd 4:0:0:0: [sdb] tag#0 Add. Sense: No additional sense information</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: xx-small;">[1962219.111844] sd 4:0:0:0: [sdb] tag#0 CDB: ATA command pass through(12)/Blank a1 06 20 da 00 00 4f c2 00 b0 00 00</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: xx-small;">[1962251.479639] JBD2: Invalid checksum recovering block 144 in log</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: xx-small;">[1962251.574260] JBD2: recovery failed</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: xx-small;">[1962251.574265] EXT4-fs (dm-4): error loading journal</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: xx-small;">[1962258.650930] JBD2: Invalid checksum recovering block 144 in log</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: xx-small;">[1962258.749283] JBD2: recovery failed</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: xx-small;">[1962258.749288] EXT4-fs (dm-4): error loading journal</span></blockquote>
</blockquote>
<br />
This generally means that there were issues with the unmount and the "checksum" for block 144 in the journal does not match the actual checksum from disk. This means there is an error reading the journal, and to prevent further corruption EXT4-fs will not mount via device mapper.<br />
<br />
So how do we fix this? If this was not encrypted you could google this error and find out to run FSCK and hopefully be done with it.<br />
<br />
Lets start with a quick reminder of the utility `lsblk`.<br />
<br />
<blockquote class="tr_bq">
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: xx-small;">root@CLCFQ92:~# lsblk --fs /dev/sdb</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: xx-small;">NAME FSTYPE LABEL UUID </span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: xx-small;">sdb </span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: xx-small;">└─sdb1 crypto_LUKS ce6cebbc-5026-4f47-9a22-da4aecfd26ad </span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: xx-small;"> └─luks-ce6cebbc-5026-4f47-9a22-da4aecfd26ad ext4 Backup e7bab3dc-87ce-4a7d-b758-34e2839b51f0 </span></blockquote>
<div>
<br /></div>
</blockquote>
This 'console' output does not render well above using the blog defaults, so I'll have to modify it for clarity, but the thing to notice is that the File System type for sdb1 is NOT ext4, it is crypto_LUKS. There is then an extended partition that is 'ext4' called luks-<uuid>. This luks partition is the one you want to run fsck on. NOT /dev/sdb1 (or /dev/sdb).</uuid><br />
<br />
So lets first open the encrypted fs...<br />
<br />
<blockquote class="tr_bq">
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">root@CLCFQ92:~# cryptsetup luksOpen /dev/sdb1 corrupted</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">Enter passphrase for /dev/sdb1: </span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"></span></blockquote>
</blockquote>
Now, lets recover the device via the name ("corrupted") we mapped to the luks partition... <br />
<blockquote class="tr_bq">
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">root@CLCFQ92:~# fsck /dev/mapper/</span><span style="font-family: "Courier New", Courier, monospace; font-size: xx-small;">corrupted</span><span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> </span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">fsck from util-linux 2.29.2</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">e2fsck 1.43.4 (31-Jan-2017)</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">Backup: recovering journal</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">JBD2: Invalid checksum recovering block 144 in log</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">Journal checksum error found in Backup</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">Backup was not cleanly unmounted, check forced.</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">Pass 1: Checking inodes, blocks, and sizes</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">Pass 2: Checking directory structure</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">Pass 3: Checking directory connectivity</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">Pass 4: Checking reference counts</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">Pass 5: Checking group summary information</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">Free blocks count wrong (436669919, counted=433657941).</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">Fix<y>? yes</y></span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">Free inodes count wrong (121861163, counted=121856397).</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">Fix<y>? yes</y></span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"><br /></span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">Backup: ***** FILE SYSTEM WAS MODIFIED *****</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">Backup: 245363/122101760 files (0.4% non-contiguous), 54719555/488377496 blocks</span></blockquote>
</blockquote>
Next, disable/close the mapped name to the luks partition:<br />
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">root@CLCFQ92:~# cryptsetup luksClose /dev/mapper/corrupted</span></blockquote>
Then mount/open the filesystem the way you normally would using gnome or whatever. Check dmesh logs for confirmation!<br />
<br />
At this point it may be good to look at using the `smartctl` utility, as your disk may be old/dying...<br />
<br />
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">root@CLCFQ92:~# smartctl -a /dev/sdb</span></blockquote>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">SMART Attributes Data Structure revision number: 10</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">Vendor Specific SMART Attributes with Thresholds:</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> 1 Raw_Read_Error_Rate 0x000f 118 100 006 Pre-fail Always - <span style="background-color: #7f6000;">200743680</span></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> 3 Spin_Up_Time 0x0003 094 093 000 Pre-fail Always - 0</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> 4 Start_Stop_Count 0x0032 100 100 020 Old_age Always - 72</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> 5 Reallocated_Sector_Ct 0x0033 100 100 036 Pre-fail Always - 0</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> 7 Seek_Error_Rate 0x000f 073 063 030 Pre-fail Always - <span style="background-color: #7f6000;">24115411</span></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> 9 Power_On_Hours 0x0032 077 077 000 Old_age Always - 20273</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> 10 Spin_Retry_Count 0x0013 100 100 097 Pre-fail Always - 0</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> 12 Power_Cycle_Count 0x0032 100 100 020 Old_age Always - 39</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">183 Runtime_Bad_Block 0x0032 100 100 000 Old_age Always - 0</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">184 End-to-End_Error 0x0032 100 100 099 Old_age Always - 0</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">187 Reported_Uncorrect 0x0032 100 100 000 Old_age Always - 0</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">188 Command_Timeout 0x0032 100 100 000 Old_age Always - 0</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">189 High_Fly_Writes 0x003a 100 100 000 Old_age Always - 0</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">190 Airflow_Temperature_Cel 0x0022 053 043 045 Old_age Always In_the_past 47 (Min/Max 42/47 #179)</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">191 G-Sense_Error_Rate 0x0032 100 100 000 Old_age Always - 0</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">192 Power-Off_Retract_Count 0x0032 100 100 000 Old_age Always - 19</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">193 Load_Cycle_Count 0x0032 100 100 000 Old_age Always - 767</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">194 Temperature_Celsius 0x0022 047 057 000 Old_age Always - 47 (0 19 0 0 0)</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">195 Hardware_ECC_Recovered 0x001a 022 021 000 Old_age Always - 200743680</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">197 Current_Pending_Sector 0x0012 100 100 000 Old_age Always - 0</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">198 Offline_Uncorrectable 0x0010 100 100 000 Old_age Offline - 0</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">199 UDMA_CRC_Error_Count 0x003e 200 200 000 Old_age Always - 0</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">240 Head_Flying_Hours 0x0000 100 253 000 Old_age Offline - 12697 (213 75 0)</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">241 Total_LBAs_Written 0x0000 100 253 000 Old_age Offline - 3453343066</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">242 Total_LBAs_Read 0x0000 100 253 000 Old_age Offline - 2807657092</span><br />
<br />
Based on the output from the SMART controller, this disk is having a lot of errors and should be backed up and disposed of.Champion Of Cyrodiilhttp://www.blogger.com/profile/06792737627887393792noreply@blogger.com1tag:blogger.com,1999:blog-6167470039524068257.post-31049146133251456882018-03-02T08:00:00.001-08:002018-03-02T10:45:56.476-08:00Easy install of NVIDIA SDK on LinuxSpent some time looking around for the best solution to this problem. I think that it is best to download the 1.8GB installer from nvidia. Then execute something like the following:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">./cuda_9.1.85_387.26_linux.run --no-drm --no-opengl-libs --extract=/home/ottch/Development/nvidia_cuda_9_1/ --toolkit --toolkitpath=/home/ottch/Development/nvidia_cuda_9_1/ --silent</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">Next you'll need to execute the cuda-linux .bin file that is generated in the toolkitpath by the last command. Ensure to set the prefix to something other than /usr/local/cuda-9.1 if you don't have sudo.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>Champion Of Cyrodiilhttp://www.blogger.com/profile/06792737627887393792noreply@blogger.com0tag:blogger.com,1999:blog-6167470039524068257.post-62068459243892393002017-09-26T08:37:00.002-07:002017-09-26T08:37:44.497-07:00Export oVirt Image/Template for VMwareoVirt supports exporting your VM or template to multiple formats. You can find a decent overview of the utility here, [[https://docs.openstack.org/image-guide/convert-images.html] https://docs.openstack.org/image-guide/convert-images.html]<br />
<br />
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> QCOW2 (KVM, Xen) - *.qcow2</span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> QED (KVM) - *.qed</span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> raw - *.raw (Sometimes no file extension provided)</span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> VDI (VirtualBox) - *.vdi</span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> VHD (Hyper-V) - *.vpc</span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> VMDK (VMware) - *.vmdk</span></blockquote>
<br />
The utility to perform Exports is 'qemu-img' with the sub-command 'convert'<br />
<br />
For example, I currently have a VM Template Image in my GlusterFS storage domain. This storage domain is mounted through the hypervisor "rosie-carreiro", so I first SSH in to the hypervisor, and 'cd' to the path where the RAW template resides.<br />
<br />
Using the oVirt admin console, I am able to identify that the template/image has a UUID of, bbb58b82-7252-41e2-9b28-72cbf74d98e4.<br />
There are other CLI tools that allow you to query storage domains and resolve UUIDs to names, however the UI is pretty intuitive for new users.<br />
<br />
When you run 'df -h' on an oVirt hypervisor, you can see the mount points used:<br />
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">pedro-godines.mycompany.com:/home/exports/iso 399G 25G 374G 7% /rhev/data-center/mnt/pedro-godines.mycompany.com:_home_exports_iso</span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;">terri-wolven.mycompany.com:/home/exports 847G 136G 711G 17% /rhev/data-center/mnt/terri-wolven.mycompany.com:_home_exports</span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;">rosie-carreiro.mycompany.com:/home/exports 8.2T 761G 7.4T 10% /rhev/data-center/mnt/rosie-carreiro.mycompany.com:_home_exports</span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;">glusterfs.mycompany.com:/gv0 1.9T 82G 1.8T 5% /rhev/data-center/mnt/glusterSD/glusterfs.mycompany.com:_gv0</span></blockquote>
<br />
In this case the Template is in the GlusterFS "gv0" volume, so I first 'cd' to /rhev/data-center/mnt/rosie-carreiro... and find the directory with the UUID, bbb58b82-7252-41e2-9b28-72cbf74d98e4.<br />
<br />
<blockquote class="tr_bq">
<span style="font-size: x-small;"> <span style="font-family: Courier New, Courier, monospace;"> [root@devops bbb58b82-7252-41e2-9b28-72cbf74d98e4]# ls -ahlt</span></span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> total 33G</span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> drwxr-xr-x. 9 36 36 4.0K Aug 25 19:08 ..</span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> drwxr-xr-x. 2 36 36 4.0K Aug 25 13:16 .</span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> -rw-r--r--. 1 36 36 318 Aug 25 13:16 4b66fd92-99a6-4651-8288-24f50fd3596a.meta</span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> -rw-rw----. 1 36 36 40G Aug 25 13:16 4b66fd92-99a6-4651-8288-24f50fd3596a</span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> -rw-rw----. 1 36 36 1.0M Aug 25 12:58 4b66fd92-99a6-4651-8288-24f50fd3596a.lease</span></blockquote>
<br />
You can also view the metadata which specifies the format and description.<br />
<br />
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> [root@devops bbb58b82-7252-41e2-9b28-72cbf74d98e4]# cat 4b66fd92-99a6-4651-8288-24f50fd3596a.meta</span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> DOMAIN=8fe3f7f3-15c9-4376-b20e-5e237b31b185</span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> CTIME=1503680317</span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> FORMAT=RAW</span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> DISKTYPE=2</span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> LEGALITY=LEGAL</span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> SIZE=83886080</span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> VOLTYPE=SHARED</span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> DESCRIPTION={"DiskAlias":"Windows7_Disk1","DiskDescription":""}</span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> IMAGE=bbb58b82-7252-41e2-9b28-72cbf74d98e4</span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> PUUID=00000000-0000-0000-0000-000000000000</span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> MTIME=0</span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> POOL_UUID=</span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> TYPE=SPARSE</span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> GEN=0</span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> EOF</span></blockquote>
<br />
You can now export the template/image to your desired format:<br />
<blockquote class="tr_bq">
<br /><span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> [root@rosie-carreiro bbb58b82-7252-41e2-9b28-72cbf74d98e4]# qemu-img convert -f raw -O vmdk -o adapter_type=lsilogic 4b66fd92-99a6-4651-8288-24f50fd3596a /root/DevImage-1.1.vmdk</span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;">Once completed, verify:</span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> [root@rosie-carreiro bbb58b82-7252-41e2-9b28-72cbf74d98e4]# ls -halt /root/DevImage-1.1.vmdk</span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> -rw-r--r--. 1 root root 33G Aug 27 16:58 /root/DevImage-1.1.vmdk</span></blockquote>
<br />
NOTE: adapter_type=lsilogic is important because qemu-img VMDK output defaults to using IDE. This means that you cannot later extend the size of the disk in VMware, as the option will actually be grayed out/disabled in vSphere. Use the lsilogic scsi controller format to support extending your disks.<br />
<div>
<br /></div>
Champion Of Cyrodiilhttp://www.blogger.com/profile/06792737627887393792noreply@blogger.com1tag:blogger.com,1999:blog-6167470039524068257.post-33092953871260839572017-08-03T13:22:00.000-07:002017-08-03T13:22:27.917-07:00Setting VirtualBox SMBIOS Settings to match your PCIn some cases you may want your VM to emulate your physical machine. For example, if you want to set the Serial Number/Service Tag to match your laptop, you can do this with the following steps:<br />
<br />
<ol>
<li>sudo dmidecode -t system</li>
<li>VBoxManage setextradata "VM name" "VBoxInternal/Devices/pcbios/0/Config/DmiSystemSerial" "#######"</li>
</ol>
<div>
In Step two above, you would not use the #######, but instead use the Serial Number displayed when running dmidecode.</div>
<div>
<br /></div>
<div>
Essentially what we are doing is configuring the SMBIOS (System Management BIOS) on the VirtualMachine to have the same strings/settings as the actual PC. This can be useful in a scenario in which you are running some software within a virtual machine that expects the hardware to be of a specific vendor or serial number.</div>
<div>
<br /></div>
<div>
More information about the SMBIOS can be found here:</div>
<div>
<span style="font-family: "Segoe UI", sans-serif; font-size: 10pt;"><a href="http://www.dmtf.org/sites/default/files/standards/documents/DSP0134_2.7.1.pdf">http://www.dmtf.org/sites/default/files/standards/documents/DSP0134_2.7.1.pdf</a></span></div>
<div>
<br /></div>
<div>
More information about setting VirtualBox VM SMBIOS data can be found here:</div>
<div>
<a href="http://www.virtualbox.org/manual/ch09.html#changedmi">http://www.virtualbox.org/manual/ch09.html#changedmi</a></div>
<div>
<br /></div>
<div>
And last, more information about how to use dmidecode on your Linux PC can be found here:</div>
<div>
<a href="https://linux.die.net/man/8/dmidecode">https://linux.die.net/man/8/dmidecode</a></div>
<div>
<br /></div>
<div>
This technique can be used to "spoof" the VM's Product ID, Serial Number, BIOS Version, Manufacturer and really any other SMBIOS value.</div>
<div>
<br /></div>
<div>
DISCLAIMER:</div>
<div>
I AM NOT RESPONSIBLE FOR ANY MISUSE OF SOFTWARE COPYRIGHT LAWS RESULTING FROM THE USE OF THIS INFORMATION</div>
<!--EndFragment -->Champion Of Cyrodiilhttp://www.blogger.com/profile/06792737627887393792noreply@blogger.com0tag:blogger.com,1999:blog-6167470039524068257.post-70677246569666092212016-08-06T09:35:00.003-07:002016-08-06T10:31:45.281-07:00Linux, Newest Kernel, Latest Hardware, Windows 10<div style="text-align: left;">
<div class="separator" style="clear: both; text-align: center;">
<a href="https://2.bp.blogspot.com/-Wfa2BhDh9k4/V6YZth729TI/AAAAAAAABb0/jR8aYLafSDk-OuyZWygee2s_qbn26cYmwCLcB/s1600/Screenshot%2Bfrom%2B2016-08-06%2B12%253A33%253A56.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="360" src="https://2.bp.blogspot.com/-Wfa2BhDh9k4/V6YZth729TI/AAAAAAAABb0/jR8aYLafSDk-OuyZWygee2s_qbn26cYmwCLcB/s640/Screenshot%2Bfrom%2B2016-08-06%2B12%253A33%253A56.png" width="640" /></a></div>
<div style="text-align: center;">
<br /></div>
<br />
About eight years ago I installed Ubuntu Intrepid Ibex, which I believe was Ubuntu 8.10. I had a Dell desktop at work and they just rolled out, "Trusted Desktop". The original intention of trusted desktop is simply to ensure an end-user's workstation is safe, free of viruses, and has the most recent security updates. This helps to keep malware and attackers out of the corporate network when users plug in to it daily.</div>
<br />
The problem for me is that because the corporation was so large, they did not ask or care about the developer's needs. The first version of trusted desktop they rolled over top of Windows XP completely removed Administration capability, as well as installed several incompatible drivers and encryption software combinations. Eventually these systems began showing the "Blue screen of death", and crawling to a halt under the poorly designed implementation of trusted platform computing.<br />
<br />
At this point I saw where Windows was going with my company and discovered they would "allow" employees to use Linux if they signed a waiver and ensured they implemented a compliant disk encryption policy. So I was off, installing Ubuntu, Virtualbox and Windows XP VM to support Windows specific work software I needed to use. In this case it was mostly Microsoft Office and the ActiveID client drivers for my smartcard to work with the VPN software.<br />
<br />
I didn't need wireless because this was a desktop plugged in at work. However I did have two outputs on my video card and hoped I would be able to run a similar dual head/monitor display, the way I had been doing on XP for a couple of years. I quickly had flashbacks of trying to get wireless drivers working on Mandrake Linux, circa 2004. It was only a bit painful getting Ibex to use my Radeon correctly, but after all of this I was content for about a year.<br />
<br />
Unfortunately, a year later support for 8.10 died out, and it was time to upgrade to 9.04, Jaunty. In that release, support for my old ATI Radeon GPU was dropped in favor of the newest version of X that was shipping w/ Jaunty. I felt betrayed by the Linux community for leaving my crappy old GPU behind. I had no idea how to write my own driver then, had my own work to do, and so I gave up and went back to Windows XP and then 7 for a long time. Of course, later I would provide systems support for about fifty developers running Ubuntu 14.04 LTS, but this topic would cause us to digress as I usually do.<br />
<br />
Well, this past week I have gone back to Linux, this time choosing Debian Jessie 8. I chose Debian because I am feeling unsure about where Canonical will be taking Ubuntu in the future, and I wanted something I was already familiar with in regard to package management. For example, I have recently used a Slackware 10 distribution on an old laptop I have, but it leaves much to be desired in regard to community support for the latest hardware.<br />
<br />
After choosing Debian for my OS, I got a new Dell Precision 7710 laptop. This thing really packs a punch with:<br />
<ul>
<li>Intel® Core™ i7-6820HQ CPU @ 2.70GHz</li>
<li>Intel Skylake GPU w/ CPU</li>
<li>Nvidia® Quadro® M3000M w/4GB GDDR5</li>
<li>32GB (4x8GB) 2133MHz DDR4 SDRAM, Non-ECC</li>
<li>Hynix 512GB M.2 PCIe NVMe Class 40 Solid State Drive</li>
</ul>
<div style="text-align: center;">
<img height="230" src="https://www.solidsolutions.co.uk/Uploaded/Image/solidworks%20Hardware/Dell-Precision-7710-LaptopWorkstation.jpg" width="400" /></div>
<br />
You can imagine my disappointment when after 8 years passing since I ran Linux on Dell for my personal system, my desktop booted with a notification that my Cinnamon desktop was being rendered in software mode. Furthermore, my wireless interface was nowhere to be found. <br />
<br />
The first issue was with a feature in the BIOS being enabled called Optimus. I don't want to make this entire post about Optimus, so I'll use a reference from Wikipedia. <i>"Nvidia Optimus is a computer GPU switching technology created by Nvidia which, depending on the resource load generated by client software applications, will seamlessly switch between two graphics adapters within a computer system in order to provide either maximum performance or minimum power draw from the system's graphics rendering hardware."</i><br />
<i><br /></i>
Apparently the Dell supported version of Ubuntu knows how to handle configuration this correctly. However, Debian Jessie out of the box could not. I chose not to attempt to enable Optimus (Optimux would have been a better name.) support on Debian, but found some interesting work here, <a href="https://nouveau.freedesktop.org/wiki/Optimus/">https://nouveau.freedesktop.org/wiki/Optimus/</a> that may assist if you choose to do so. For me, I have simply disabled the setting in the BIOS, and have opted for the NVidia GPU to be my primary display adapter. Perhaps later I will investigate Optimus further.<br />
<br />
However this led me to my next issue, in that my GPU is not yet supported by Nouveau. The Quadro M3000M is part of the Maxwell-2 series.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-AUJfIFypMuI/V6YMIMiCZWI/AAAAAAAABbQ/IxLVPEKvcwc6hvr0GWUG59evVDeHkuGAgCLcB/s1600/Quadro_M5000M_678x452.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="352" src="https://1.bp.blogspot.com/-AUJfIFypMuI/V6YMIMiCZWI/AAAAAAAABbQ/IxLVPEKvcwc6hvr0GWUG59evVDeHkuGAgCLcB/s640/Quadro_M5000M_678x452.jpg" width="640" /></a></div>
<div style="text-align: center;">
<br /></div>
<div style="text-align: center;">
</div>
<table align="center" border="0" cellpadding="0" cellspacing="1" style="-webkit-text-stroke: 1px rgba(0, 0, 0, 0); background: none rgb(246, 246, 246); border-collapse: collapse; border-spacing: 0px; border: none; box-sizing: border-box; color: #444444; font-family: Arimo, sans-serif; font-size: 14px; font-stretch: inherit; font-variant-numeric: inherit; line-height: 21px; margin: auto; max-width: none; padding: 0px; vertical-align: baseline; width: 640px;"><tbody style="border: 0px; box-sizing: border-box; font-family: inherit; font-size: inherit; font-stretch: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: inherit; margin: 0px; padding: 0px; vertical-align: baseline;">
<tr class="tgrey" style="border: 0px; box-sizing: border-box; font-family: inherit; font-size: inherit; font-stretch: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: inherit; margin: 0px; padding: 0px; vertical-align: baseline;"><td align="center" colspan="6" style="background: none 0px 0px repeat scroll rgb(132, 132, 132) !important; border-bottom: 1px solid rgb(214, 214, 214); border-left: 1px solid rgb(255, 255, 255); border-right: none; border-top: none; box-sizing: border-box; color: white; font-family: Rokkitt, serif; font-size: 21px; font-stretch: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; height: 46px; line-height: normal; margin: 1px; padding: 8px 0px; text-align: center; vertical-align: top;">NVIDIA Quadro Mobile Specification Comparison (High-End)</td></tr>
<tr class="tlblue" style="background: none 0px 0px repeat scroll rgb(0, 0, 0) !important; border-bottom-color: rgb(214, 214, 214); border-bottom-style: solid; border-width: 0px 0px 1px; box-sizing: border-box; color: #33d5d5; font-family: Rokkitt, serif; font-size: 21px; font-stretch: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; height: 46px; line-height: inherit; margin: 0px 0px 1px; padding: 8px 0px; text-align: center; vertical-align: baseline;"><td style="background: none 0px 0px repeat scroll rgb(34, 149, 171) !important; border-bottom: 1px solid rgb(214, 214, 214); border-left: 1px solid rgb(255, 255, 255); border-right: none; border-top: none; box-sizing: border-box; color: white; font-size: 17px; font-stretch: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; height: 46px; line-height: normal; margin: 1px; padding: 6px; vertical-align: top;" width="181"></td><td align="center" style="background: none 0px 0px repeat scroll rgb(34, 149, 171) !important; border-bottom: 1px solid rgb(214, 214, 214); border-left: 1px solid rgb(255, 255, 255); border-right: none; border-top: none; box-sizing: border-box; color: white; font-size: 17px; font-stretch: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; height: 46px; line-height: normal; margin: 1px; padding: 6px; vertical-align: top;" valign="middle" width="152">Quadro M5000M</td><td align="center" style="background: none 0px 0px repeat scroll rgb(34, 149, 171) !important; border-bottom: 1px solid rgb(214, 214, 214); border-left: 1px solid rgb(255, 255, 255); border-right: none; border-top: none; box-sizing: border-box; color: white; font-size: 17px; font-stretch: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; height: 46px; line-height: normal; margin: 1px; padding: 6px; vertical-align: top;" valign="middle" width="151">Quadro M4000M</td><td align="center" style="background: none 0px 0px repeat scroll rgb(34, 149, 171) !important; border-bottom: 1px solid rgb(214, 214, 214); border-left: 1px solid rgb(255, 255, 255); border-right: none; border-top: none; box-sizing: border-box; color: white; font-size: 17px; font-stretch: inherit; font-style: inherit; font-variant: inherit; height: 46px; line-height: normal; margin: 1px; padding: 6px; vertical-align: top;" valign="middle" width="151"><b>Quadro M3000M</b></td></tr>
<tr style="border: 0px; box-sizing: border-box; font-family: inherit; font-size: inherit; font-stretch: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: inherit; margin: 0px; padding: 0px; vertical-align: baseline;"><td class="tlgrey" style="background: rgb(238, 238, 238); border-bottom: 1px solid rgb(255, 255, 255); border-left: 1px solid rgb(255, 255, 255); border-right: none; border-top: none; box-sizing: border-box; font-family: inherit; font-size: inherit; font-stretch: inherit; font-style: inherit; font-variant: inherit; font-weight: bold; line-height: normal; margin: 1px; padding: 7px; vertical-align: top;">CUDA Cores</td><td align="center" style="background: rgb(238, 238, 238); border-bottom: 1px solid rgb(255, 255, 255); border-left: 1px solid rgb(255, 255, 255); border-right: none; border-top: none; box-sizing: border-box; font-family: inherit; font-size: inherit; font-stretch: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: normal; margin: 1px; padding: 7px; vertical-align: top;" valign="middle">1536</td><td align="center" style="background: rgb(238, 238, 238); border-bottom: 1px solid rgb(255, 255, 255); border-left: 1px solid rgb(255, 255, 255); border-right: none; border-top: none; box-sizing: border-box; font-family: inherit; font-size: inherit; font-stretch: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: normal; margin: 1px; padding: 7px; vertical-align: top;" valign="middle">1280</td><td align="center" style="background: rgb(238, 238, 238); border-bottom: 1px solid rgb(255, 255, 255); border-left: 1px solid rgb(255, 255, 255); border-right: none; border-top: none; box-sizing: border-box; font-family: inherit; font-size: inherit; font-stretch: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: normal; margin: 1px; padding: 7px; vertical-align: top;" valign="middle">1024</td></tr>
<tr style="border: 0px; box-sizing: border-box; font-family: inherit; font-size: inherit; font-stretch: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: inherit; margin: 0px; padding: 0px; vertical-align: baseline;"><td class="tlgrey" style="background: rgb(238, 238, 238); border-bottom: 1px solid rgb(255, 255, 255); border-left: 1px solid rgb(255, 255, 255); border-right: none; border-top: none; box-sizing: border-box; font-family: inherit; font-size: inherit; font-stretch: inherit; font-style: inherit; font-variant: inherit; font-weight: bold; line-height: normal; margin: 1px; padding: 7px; vertical-align: top;">Memory Clock</td><td align="center" style="background: rgb(238, 238, 238); border-bottom: 1px solid rgb(255, 255, 255); border-left: 1px solid rgb(255, 255, 255); border-right: none; border-top: none; box-sizing: border-box; font-family: inherit; font-size: inherit; font-stretch: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: normal; margin: 1px; padding: 7px; vertical-align: top;" valign="middle">5GHz GDDR5</td><td align="center" style="background: rgb(238, 238, 238); border-bottom: 1px solid rgb(255, 255, 255); border-left: 1px solid rgb(255, 255, 255); border-right: none; border-top: none; box-sizing: border-box; font-family: inherit; font-size: inherit; font-stretch: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: normal; margin: 1px; padding: 7px; vertical-align: top;" valign="middle">5GHz GDDR5</td><td align="center" style="background: rgb(238, 238, 238); border-bottom: 1px solid rgb(255, 255, 255); border-left: 1px solid rgb(255, 255, 255); border-right: none; border-top: none; box-sizing: border-box; font-family: inherit; font-size: inherit; font-stretch: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: normal; margin: 1px; padding: 7px; vertical-align: top;" valign="middle">5GHz GDDR5</td></tr>
<tr style="border: 0px; box-sizing: border-box; font-family: inherit; font-size: inherit; font-stretch: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: inherit; margin: 0px; padding: 0px; vertical-align: baseline;"><td class="tlgrey" style="background: rgb(238, 238, 238); border-bottom: 1px solid rgb(255, 255, 255); border-left: 1px solid rgb(255, 255, 255); border-right: none; border-top: none; box-sizing: border-box; font-family: inherit; font-size: inherit; font-stretch: inherit; font-style: inherit; font-variant: inherit; font-weight: bold; line-height: normal; margin: 1px; padding: 7px; vertical-align: top;">Memory Bus Width</td><td align="center" style="background: rgb(238, 238, 238); border-bottom: 1px solid rgb(255, 255, 255); border-left: 1px solid rgb(255, 255, 255); border-right: none; border-top: none; box-sizing: border-box; font-family: inherit; font-size: inherit; font-stretch: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: normal; margin: 1px; padding: 7px; vertical-align: top;" valign="middle">256-bit</td><td align="center" style="background: rgb(238, 238, 238); border-bottom: 1px solid rgb(255, 255, 255); border-left: 1px solid rgb(255, 255, 255); border-right: none; border-top: none; box-sizing: border-box; font-family: inherit; font-size: inherit; font-stretch: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: normal; margin: 1px; padding: 7px; vertical-align: top;" valign="middle">256-bit</td><td align="center" style="background: rgb(238, 238, 238); border-bottom: 1px solid rgb(255, 255, 255); border-left: 1px solid rgb(255, 255, 255); border-right: none; border-top: none; box-sizing: border-box; font-family: inherit; font-size: inherit; font-stretch: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: normal; margin: 1px; padding: 7px; vertical-align: top;" valign="middle">256-bit</td></tr>
<tr style="border: 0px; box-sizing: border-box; font-family: inherit; font-size: inherit; font-stretch: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: inherit; margin: 0px; padding: 0px; vertical-align: baseline;"><td class="tlgrey" style="background: rgb(238, 238, 238); border-bottom: 1px solid rgb(255, 255, 255); border-left: 1px solid rgb(255, 255, 255); border-right: none; border-top: none; box-sizing: border-box; font-family: inherit; font-size: inherit; font-stretch: inherit; font-style: inherit; font-variant: inherit; font-weight: bold; line-height: normal; margin: 1px; padding: 7px; vertical-align: top;">VRAM</td><td align="center" style="background: rgb(238, 238, 238); border-bottom: 1px solid rgb(255, 255, 255); border-left: 1px solid rgb(255, 255, 255); border-right: none; border-top: none; box-sizing: border-box; font-family: inherit; font-size: inherit; font-stretch: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: normal; margin: 1px; padding: 7px; vertical-align: top;" valign="middle">8GB</td><td align="center" style="background: rgb(238, 238, 238); border-bottom: 1px solid rgb(255, 255, 255); border-left: 1px solid rgb(255, 255, 255); border-right: none; border-top: none; box-sizing: border-box; font-family: inherit; font-size: inherit; font-stretch: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: normal; margin: 1px; padding: 7px; vertical-align: top;" valign="middle">4GB</td><td align="center" style="background: rgb(238, 238, 238); border-bottom: 1px solid rgb(255, 255, 255); border-left: 1px solid rgb(255, 255, 255); border-right: none; border-top: none; box-sizing: border-box; font-family: inherit; font-size: inherit; font-stretch: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: normal; margin: 1px; padding: 7px; vertical-align: top;" valign="middle">4GB</td></tr>
<tr style="border: 0px; box-sizing: border-box; font-family: inherit; font-size: inherit; font-stretch: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: inherit; margin: 0px; padding: 0px; vertical-align: baseline;"><td class="tlgrey" style="background: rgb(238, 238, 238); border-bottom: 1px solid rgb(255, 255, 255); border-left: 1px solid rgb(255, 255, 255); border-right: none; border-top: none; box-sizing: border-box; font-family: inherit; font-size: inherit; font-stretch: inherit; font-style: inherit; font-variant: inherit; font-weight: bold; line-height: normal; margin: 1px; padding: 7px; vertical-align: top;">FP64</td><td align="center" style="background: rgb(238, 238, 238); border-bottom: 1px solid rgb(255, 255, 255); border-left: 1px solid rgb(255, 255, 255); border-right: none; border-top: none; box-sizing: border-box; font-family: inherit; font-size: inherit; font-stretch: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: normal; margin: 1px; padding: 7px; vertical-align: top;" valign="middle">1/32</td><td align="center" style="background: rgb(238, 238, 238); border-bottom: 1px solid rgb(255, 255, 255); border-left: 1px solid rgb(255, 255, 255); border-right: none; border-top: none; box-sizing: border-box; font-family: inherit; font-size: inherit; font-stretch: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: normal; margin: 1px; padding: 7px; vertical-align: top;" valign="middle">1/32</td><td align="center" style="background: rgb(238, 238, 238); border-bottom: 1px solid rgb(255, 255, 255); border-left: 1px solid rgb(255, 255, 255); border-right: none; border-top: none; box-sizing: border-box; font-family: inherit; font-size: inherit; font-stretch: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: normal; margin: 1px; padding: 7px; vertical-align: top;" valign="middle">1/32</td></tr>
<tr style="border: 0px; box-sizing: border-box; font-family: inherit; font-size: inherit; font-stretch: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: inherit; margin: 0px; padding: 0px; vertical-align: baseline;"><td class="tlgrey" style="background: rgb(238, 238, 238); border-bottom: 1px solid rgb(255, 255, 255); border-left: 1px solid rgb(255, 255, 255); border-right: none; border-top: none; box-sizing: border-box; font-family: inherit; font-size: inherit; font-stretch: inherit; font-style: inherit; font-variant: inherit; font-weight: bold; line-height: normal; margin: 1px; padding: 7px; vertical-align: top;">TDP</td><td align="center" style="background: rgb(238, 238, 238); border-bottom: 1px solid rgb(255, 255, 255); border-left: 1px solid rgb(255, 255, 255); border-right: none; border-top: none; box-sizing: border-box; font-family: inherit; font-size: inherit; font-stretch: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: normal; margin: 1px; padding: 7px; vertical-align: top;" valign="middle">100W</td><td align="center" style="background: rgb(238, 238, 238); border-bottom: 1px solid rgb(255, 255, 255); border-left: 1px solid rgb(255, 255, 255); border-right: none; border-top: none; box-sizing: border-box; font-family: inherit; font-size: inherit; font-stretch: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: normal; margin: 1px; padding: 7px; vertical-align: top;" valign="middle">100W</td><td align="center" style="background: rgb(238, 238, 238); border-bottom: 1px solid rgb(255, 255, 255); border-left: 1px solid rgb(255, 255, 255); border-right: none; border-top: none; box-sizing: border-box; font-family: inherit; font-size: inherit; font-stretch: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: normal; margin: 1px; padding: 7px; vertical-align: top;" valign="middle">75W</td></tr>
<tr style="border: 0px; box-sizing: border-box; font-family: inherit; font-size: inherit; font-stretch: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: inherit; margin: 0px; padding: 0px; vertical-align: baseline;"><td class="tlgrey" style="background: rgb(238, 238, 238); border-bottom: 1px solid rgb(255, 255, 255); border-left: 1px solid rgb(255, 255, 255); border-right: none; border-top: none; box-sizing: border-box; font-family: inherit; font-size: inherit; font-stretch: inherit; font-style: inherit; font-variant: inherit; font-weight: bold; line-height: normal; margin: 1px; padding: 7px; vertical-align: top;">GPU</td><td align="center" style="background: rgb(238, 238, 238); border-bottom: 1px solid rgb(255, 255, 255); border-left: 1px solid rgb(255, 255, 255); border-right: none; border-top: none; box-sizing: border-box; font-family: inherit; font-size: inherit; font-stretch: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: normal; margin: 1px; padding: 7px; vertical-align: top;" valign="middle">GM204</td><td align="center" style="background: rgb(238, 238, 238); border-bottom: 1px solid rgb(255, 255, 255); border-left: 1px solid rgb(255, 255, 255); border-right: none; border-top: none; box-sizing: border-box; font-family: inherit; font-size: inherit; font-stretch: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: normal; margin: 1px; padding: 7px; vertical-align: top;" valign="middle">GM204</td><td align="center" style="background: rgb(238, 238, 238); border-bottom: 1px solid rgb(255, 255, 255); border-left: 1px solid rgb(255, 255, 255); border-right: none; border-top: none; box-sizing: border-box; font-family: inherit; font-size: inherit; font-stretch: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: normal; margin: 1px; padding: 7px; vertical-align: top;" valign="middle">GM204</td></tr>
<tr style="border: 0px; box-sizing: border-box; font-family: inherit; font-size: inherit; font-stretch: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: inherit; margin: 0px; padding: 0px; vertical-align: baseline;"><td class="tlgrey" style="background: rgb(238, 238, 238); border-bottom: 1px solid rgb(255, 255, 255); border-left: 1px solid rgb(255, 255, 255); border-right: none; border-top: none; box-sizing: border-box; font-family: inherit; font-size: inherit; font-stretch: inherit; font-style: inherit; font-variant: inherit; font-weight: bold; line-height: normal; margin: 1px; padding: 7px; vertical-align: top;">Architecture</td><td align="center" style="background: rgb(238, 238, 238); border-bottom: 1px solid rgb(255, 255, 255); border-left: 1px solid rgb(255, 255, 255); border-right: none; border-top: none; box-sizing: border-box; font-family: inherit; font-size: inherit; font-stretch: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: normal; margin: 1px; padding: 7px; vertical-align: top;" valign="middle">Maxwell 2</td><td align="center" style="background: rgb(238, 238, 238); border-bottom: 1px solid rgb(255, 255, 255); border-left: 1px solid rgb(255, 255, 255); border-right: none; border-top: none; box-sizing: border-box; font-family: inherit; font-size: inherit; font-stretch: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: normal; margin: 1px; padding: 7px; vertical-align: top;" valign="middle">Maxwell 2</td><td align="center" style="background: rgb(238, 238, 238); border-bottom: 1px solid rgb(255, 255, 255); border-left: 1px solid rgb(255, 255, 255); border-right: none; border-top: none; box-sizing: border-box; font-family: inherit; font-size: inherit; font-stretch: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: normal; margin: 1px; padding: 7px; vertical-align: top;" valign="middle">Maxwell 2</td></tr>
</tbody></table>
<br />
<div>
A quick review of the list at <a href="https://nouveau.freedesktop.org/wiki/CodeNames/">https://nouveau.freedesktop.org/wiki/CodeNames/</a> will show you that this GPU is not supported. So, I had to install the bundle available from Nvidia. The important thing to remember when bundling the manufacture's driver is to ensure DKMS (Dynamic Kernel Module Support) is installed and that Nvidia uses DKMS to manage the kernel module between kernel updates.</div>
<div>
<br /></div>
<div>
So with my GPU now working, it was time to figure out the issue with my wireless. A quick `lspci | grep Wireless` resolved,<br />
<br /></div>
<div>
<span style="background-color: black; color: lime; font-family: "courier new" , "courier" , monospace;">02:00.0 Network controller: Intel Corporation Wireless 8260 (rev 3a)</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div>
<div>
The firmware for Intel wireless adapters generally ends up in a package called, 'iwlwifi' and once installed on most systems, will match the vendor id on your hardware during modprobe, and load the correct module. As luck would have it, this card's module was not added/patched until November 2015.</div>
<div>
<a href="https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/drivers/net/wireless/iwlwifi/pcie/drv.c?id=4ab75944c4b324c1f5f01dbd4c4d122d2b9da187">https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/drivers/net/wireless/iwlwifi/pcie/drv.c?id=4ab75944c4b324c1f5f01dbd4c4d122d2b9da187</a></div>
<div>
<br /></div>
<div>
This means my driver doesn't really show up until the November 2015 timeframe, which is kernel 4.3. Turns out Debian has backports up to Kernel 4.6! So, upgrading with the backports should include the firmware for my wifi card. Backports warns you to only update a package if you NEED it, and encourages you not to upgrade everything. However, so much depends on the Linux kernel, that upgrading that specific package creates a dependency for many other backport requied packages. As a result, I ended up using backports for pretty much all of my software. Which is Virtualbox, X, and Gnome3.</div>
<div>
<br /></div>
<div>
To perform this upgrade, first gloss over, <a href="https://backports.debian.org/Instructions/">https://backports.debian.org/Instructions/</a>. Then, after you have added the backports repo, run the following,</div>
<div>
<br /></div>
<div>
<span style="background-color: black; color: lime; font-family: "courier new" , "courier" , monospace;">$ sudo apt-get -t jessie-backports install linux-image-amd64</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div>
<div>
Once this is complete, rebooting your PC should prompt you for the latest kernel at the Debian GRUB screen, in my case this is 4.6+74~bpo8+1. If I understand this naming convention correctly, it means</div>
<div>
<b>"Linux Kernel 4.6, Build number 74, Backport for 8.0 Build 1"</b>. Which is quite the mouthful.</div>
<div>
<br /></div>
<div>
As a bonus, the above kernel will also include support for the latest bluetooth adapter! The end result of all this work was a bleeding edge opensource operating system using Virtualbox 5.0 (from backports) which has USB 3.0 support and 3D acceleration!<br />
<br /></div>
<div style="text-align: center;">
<br /></div>
Champion Of Cyrodiilhttp://www.blogger.com/profile/06792737627887393792noreply@blogger.com0tag:blogger.com,1999:blog-6167470039524068257.post-58006747088820057462016-07-28T12:22:00.001-07:002016-08-06T10:41:20.482-07:00Parsing tcpdump/pcap files in bash, without Wireshark.Sometimes I work in an environment in which software access is restricted. Things like Wireshark are DEFINITELY not allowed.<br />
<br />
However, I'm still expected to troubleshoot network connectivity issues. Luckily, tcpdump exists in the EPEL Repo, but leaves much to be desired when trying to read the PCAP files. I decided to read up on the PCAP spec, and create my own parser in bash. Crazy? Maybe. But, it does work.<br />
<br />
Another nice capability here is that using something like vim, you can easily tweak this script to dump application data for further debugging. For example, switch your HTTPd to run in Non-SSL, run a capture, and dump the application data. Of course, you may have to work out the correct byte offset for your packets. As this is a bare minimal implementation of a pcap parser and does not nearly support the full scope of capability provided by libpcap.<br />
<br />
The real issue here is that I have not taken the time to parse some flags which consist of single bits. There is also much work to be done in regard to writing the conditionals for various Ethernet, IP Routing and TCP frames. But this is a good baseline to start.<br />
<br />
To use this script just run 'tcpdump -w file.pcap -X' to run a capture. Once done, pass the file as an argument to the pcap-analyzer bash script, and you will be able to read your frames on the CLI! This should work out of the box with Centos/RHEL 6 Minimal, with the addition of the tcpdump RPM.<br />
<br />
Here is a screenshot.<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://4.bp.blogspot.com/-qxCGpuZkEsU/V5pWsPWPbII/AAAAAAAABZY/YaOqOCmFQowIL8mqMFCr3Ig510znGIgPQCLcB/s1600/pcap.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="574" src="https://4.bp.blogspot.com/-qxCGpuZkEsU/V5pWsPWPbII/AAAAAAAABZY/YaOqOCmFQowIL8mqMFCr3Ig510znGIgPQCLcB/s640/pcap.png" width="640" /></a></div>
<br />
As you can see frame 2 starts to break because of the parsing. Some flags actually mean, the header is going to be 28 bytes instead of 26. As a result of the byte variations, the byte iterator ends up on some obscure part of the stream I don't handle, and things get start to break. Git forks are welcome if you like this little utility and are interested in making it more robust.<br />
<br />
<a href="https://github.com/charlescva/common/blob/master/pcap-analyzer.sh">https://github.com/charlescva/common/blob/master/pcap-analyzer.sh</a><br />
<br />
One final important characteristic about Ethernet wire data is that it is ALWAYS Big Endian. This is confusing because most CPUs are little endian. To work around this issue, I use both 'hexdump' and 'od' because one enforces big endian while the other is CPU dependent.<br />
<br />
Other resources Used:<br />
http://www.tcpdump.org/manpages/pcap-savefile.5.html<br />
http://www.tcpdump.org/linktypes.html<br />
https://en.wikipedia.org/wiki/Ethernet_frame#Ethernet_II<br />
https://en.wikipedia.org/wiki/IPv4#Packet_structure<br />
https://en.wikipedia.org/wiki/Transmission_Control_Protocol#TCP_segment_structure<br />
<br />Champion Of Cyrodiilhttp://www.blogger.com/profile/06792737627887393792noreply@blogger.com0tag:blogger.com,1999:blog-6167470039524068257.post-12762423962558187652016-06-11T11:28:00.001-07:002016-07-28T11:51:12.301-07:00Apache Nifi Behind Apache (httpd) (SSL to Non-SSL)ProxyPreserveHost On<br />
<br />
<location nifi=""></location><br />
ProxyPass "http://localhost:8080/nifi" max=20 ttl=120 retry=300<br />
ProxyPassReverse "http://localhost:8080/nifi"<br />
RequestHeader add X-ProxyScheme "https"<br />
RequestHeader add X-ProxyHost "proxyserver.acme.com"<br />
RequestHeader add X-ProxyPort "443"<br />
<br />
<br />
<location nifi-api=""></location><br />
ProxyPass "http://localhost:8080/nifi-api" max=20 ttl=120 retry=300<br />
ProxyPassReverse "http://localhost:8080/nifi-api"<br />
RequestHeader add X-ProxyScheme "https"<br />
RequestHeader add X-ProxyHost "proxyserver.acme.com"<br />
RequestHeader add X-ProxyPort "443"<br />
<br />
<br />
<location nifi-docs=""></location><br />
ProxyPass "http://localhost:8080/nifi-docs" max=20 ttl=120 retry=300<br />
ProxyPassReverse "http://localhost:8080/nifi-docs"<br />
RequestHeader add X-ProxyScheme "https"<br />
RequestHeader add X-ProxyHost "proxyserver.acme.com"<br />
RequestHeader add X-ProxyPort "443"<br />
<br />
<div>
<br /></div>
Champion Of Cyrodiilhttp://www.blogger.com/profile/06792737627887393792noreply@blogger.com0tag:blogger.com,1999:blog-6167470039524068257.post-60790117151018018022016-04-21T21:42:00.001-07:002016-04-26T10:21:15.661-07:00NAB Show 2016<div style="text-align: left;">
NABShow 2016 was a great experience. I have never taken interest in the production of movies, radio, or any form of media in my life, but this event has changed my perspective dramatically. This will be my first in a series of posts, in which I will share photos and information I learned throughout this event.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
A recent gig has required me to take an interest in video production. It is not always common to have an employer pay for the attendance of a conference, but I think it is important. If you take away anything from this post, it should be that self-enrichment is required to succeed in any field. And you should push your employer to give you an opportunity to pursue it.</div>
<div style="text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<a href="https://2.bp.blogspot.com/-3O_ziOiSaxU/Vxmf1xzUmiI/AAAAAAAABMM/Vdy3QENvLqkeOou_lG6BrjiefgDyoRYKwCLcB/s1600/20160419_063933.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="360" src="https://2.bp.blogspot.com/-3O_ziOiSaxU/Vxmf1xzUmiI/AAAAAAAABMM/Vdy3QENvLqkeOou_lG6BrjiefgDyoRYKwCLcB/s640/20160419_063933.jpg" width="640" /></a></div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
The show was in Las Vegas, Nevada. Being from Virginia, it was a long flight. However, after meeting so many people from Europe and China, I cannot complain. I stayed at a cheaper Hotel/Casino called "<i>Circus Circus</i>". </div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
The "<i>Hotel and Casino</i>" was dated and odd with the whole circus theme, but the room was clean and the last night I was there, I had fun losing a few dollars at the slot machines and roulette. They do however have a good Italian place with the best meatball sub, and a great steakhouse that has class and has been around for years. <b>ProTip: The steakhouse is reservation only, however if you are alone, you can eat at the classy bar.</b> Also, I can say that I appreciate not seeing one single clown the whole time I was there. I think acrobats and dancers have mostly replaced them.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
The event is Monday through Thursday every year. I attended Tues & Wed, and had enough for my first visit. However, if I get the chance to go again, I'll plan it better. The hotel is only a 15 minute walk from the convention center. The Las Vegas Convention Center is HUGE. To give you an idea of what I mean, the "Westgate" is 200,000 sq ft (19,000 m2) and has it's own station on the Vegas monorail.</div>
<div style="text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<a href="https://3.bp.blogspot.com/-P0X5WjYCnF0/VxmertwI1uI/AAAAAAAABMA/mVoaLdGWPpwNQfTMyLGFmILZzwXLzIffgCLcB/s1600/VegasConventionCenter.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="273" src="https://3.bp.blogspot.com/-P0X5WjYCnF0/VxmertwI1uI/AAAAAAAABMA/mVoaLdGWPpwNQfTMyLGFmILZzwXLzIffgCLcB/s640/VegasConventionCenter.png" width="640" /></a></div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
As you can see the convention center is much larger than the Westgate, located to the left of the blue "North Hall" in the picture above. So be prepared to walk several MILES at this show.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
There were so many exhibitors I could not count them all. But I learned a lot from some of the free sessions that were included with the Exhibits only pass. Make sure you go to those while you are there. They are usually only 45 minutes, but worth the time.</div>
<div style="text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://4.bp.blogspot.com/-jKK01h2PFoI/VxmhXxLUfeI/AAAAAAAABMY/H4aw-FRBjfgZSPyTwoRE60QPYe8EOwk6ACLcB/s1600/20160419_092600.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="360" src="https://4.bp.blogspot.com/-jKK01h2PFoI/VxmhXxLUfeI/AAAAAAAABMY/H4aw-FRBjfgZSPyTwoRE60QPYe8EOwk6ACLcB/s640/20160419_092600.jpg" width="640" /></a></div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
I realized I was at a whole new type of tech event when the first thing I came across was being called "IP Hybrid Routing". My first thought was, "What to they mean by 'hybrid'? Is there some other kind of digital routing that is more efficient than TCP/IP that I don't know about?" </div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
Turns out there is. Video and audio signals have been around a lot longer than the internet. These original analog signals gave birth to the broadcasting industry. As a result, the industry vendors behind analog equipment went straight to using hardware components. Custom hardware for digital signal processing was faster than computer processors (or GPUs), and could perform the digital operations as quickly as they were needed to encode and decode the signals being recorded live.</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://d2wfckeh0j530f.cloudfront.net/wordpress/wp-content/uploads/2015/04/Avenir-Mini-3_updated-workflow-400x267.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://d2wfckeh0j530f.cloudfront.net/wordpress/wp-content/uploads/2015/04/Avenir-Mini-3_updated-workflow-400x267.png" height="213" width="320" /></a></div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
As a result, in 2016 there are thousands of vendors of this equipment worldwide. And they are all chomping at the bit to crack in to the cloud.</div>
<div style="text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-N4mk7AE9PGQ/Vxmk2eF4bgI/AAAAAAAABM0/ZQ06OzcTMRck7cvb7YasMxZCEpWjevfOACLcB/s1600/20160419_143229.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="360" src="https://1.bp.blogspot.com/-N4mk7AE9PGQ/Vxmk2eF4bgI/AAAAAAAABM0/ZQ06OzcTMRck7cvb7YasMxZCEpWjevfOACLcB/s640/20160419_143229.jpg" width="640" /></a></div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
I attended two sessions by <a href="https://github.com/JoshuaKolden">Josh Kolden</a> regarding his C4 ID system. His system essentially uses natural language semantics and byte hashes to create a metadata tagging system for all of the artifacts used in video production. There is a whitepaper that goes in to more detail available at his site, <a href="http://cccc.io/">http://cccc.io</a>. He also stresses the importance of using a RESTful API with JSON as the glue between your software components. He named several but <a href="http://www.avid.com/">Avid</a> and <a href="https://www.thefoundry.co.uk/products/nuke/">Nuke</a> are two that come to mind. If you are in to software engineering, it shouldn't take much to wrap your mind around how quickly the cloud is going to (or already is) provide a major role in the future of media production. The C4 ID system was used to produce "<a href="http://www.imdb.com/title/tt5258302/">The Suitcase</a>" a movie currently in post-production and will be released later this year. It was very interesting to hear about the challenges getting this metadata collection process integrated with the movie production staff.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
I spent the rest of my first day looking at Drones, LED displays and other gadgets. There was a TON of them.</div>
<div style="text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-kUvylAc9ipI/Vxmo3trQVEI/AAAAAAAABNg/WsolBOkQ5ewm2w03ttpEoljZlOvbqgPNwCLcB/s1600/DualPropDrone.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="360" src="https://1.bp.blogspot.com/-kUvylAc9ipI/Vxmo3trQVEI/AAAAAAAABNg/WsolBOkQ5ewm2w03ttpEoljZlOvbqgPNwCLcB/s640/DualPropDrone.jpg" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
Dual-Prop Airplane Drone</div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-88S9oZqBSxI/VxmngrQI4cI/AAAAAAAABNI/CQOLhfFR8mIjHTqwUmUMse2bdbo1YdrpACLcB/s1600/20160419_104537.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="400" src="https://1.bp.blogspot.com/-88S9oZqBSxI/VxmngrQI4cI/AAAAAAAABNI/CQOLhfFR8mIjHTqwUmUMse2bdbo1YdrpACLcB/s400/20160419_104537.jpg" width="225" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
Stabilized Sensor Suite</div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://2.bp.blogspot.com/-q-veSLDBABg/VxmoPLGtXbI/AAAAAAAABNQ/WK_ZDP1a5zsG_LaNTz3G9bZqJIHgEe7FgCLcB/s1600/CarbonFIberCameraDrone.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="456" src="https://2.bp.blogspot.com/-q-veSLDBABg/VxmoPLGtXbI/AAAAAAAABNQ/WK_ZDP1a5zsG_LaNTz3G9bZqJIHgEe7FgCLcB/s640/CarbonFIberCameraDrone.jpg" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
Octo-copter with gimbals and professional video camera</div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://2.bp.blogspot.com/-Bs3RZ44lQ9Y/Vxmp4wcaqEI/AAAAAAAABNw/HaZNOUZ_gFkhnU_McMFBqvGud33NQhCOQCLcB/s1600/AdjustableCarbonFiber.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="320" src="https://2.bp.blogspot.com/-Bs3RZ44lQ9Y/Vxmp4wcaqEI/AAAAAAAABNw/HaZNOUZ_gFkhnU_McMFBqvGud33NQhCOQCLcB/s320/AdjustableCarbonFiber.jpg" width="244" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
With adjustable carbon-fiber chasis</div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://3.bp.blogspot.com/-FDsCf7BU5WA/VxmqEmPHTiI/AAAAAAAABN0/76_O2GSSl0ANp2NyrAn46Kjdv-yt8uT1ACLcB/s1600/GiantLEDDisplay.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="360" src="https://3.bp.blogspot.com/-FDsCf7BU5WA/VxmqEmPHTiI/AAAAAAAABN0/76_O2GSSl0ANp2NyrAn46Kjdv-yt8uT1ACLcB/s640/GiantLEDDisplay.jpg" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
These enormous LED displays are high resolution at a distance, and can be scaled to any size</div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://3.bp.blogspot.com/-KyAOpIDinDA/VxmqlSdLsNI/AAAAAAAABN8/bBGCtpIC_dQC3u95CILg3u4L8F9nDxZlQCLcB/s1600/20160420_091158.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="180" src="https://3.bp.blogspot.com/-KyAOpIDinDA/VxmqlSdLsNI/AAAAAAAABN8/bBGCtpIC_dQC3u95CILg3u4L8F9nDxZlQCLcB/s320/20160420_091158.jpg" width="320" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
The LED screens are an array of squares with an input, output and LCD display showing about 14 volts and 37 degrees Celsius.</div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://2.bp.blogspot.com/-2XrXJ8rQ_o0/VxmrK9snnqI/AAAAAAAABOE/Xq2jlNr6i8sdchaxuStUON0qDyGKprspQCLcB/s1600/LEDCubes.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="320" src="https://2.bp.blogspot.com/-2XrXJ8rQ_o0/VxmrK9snnqI/AAAAAAAABOE/Xq2jlNr6i8sdchaxuStUON0qDyGKprspQCLcB/s320/LEDCubes.jpg" width="208" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
You can even make cubes with live video for your nightclub.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
In my next post, I will go over more equipment and talk a bit more about the current VR industry, the future of VR, and post more photos of the event.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<a href="http://lmgtfy.com/?q=NABShow">http://lmgtfy.com/?q=NABShow</a></div>
Champion Of Cyrodiilhttp://www.blogger.com/profile/06792737627887393792noreply@blogger.com0tag:blogger.com,1999:blog-6167470039524068257.post-62802246979736940882016-03-23T08:49:00.003-07:002016-03-23T08:49:49.053-07:00QuickTip: Tree AliasSometimes directory structures are a bit of a nest. You often want to see the full structure, but may not have access to packages like 'tree'. I found a great expression and decided to wrap it into an aliased command in ~/.bash_profile<br />
<br />
alias tree="ls -R | grep \":$\" | sed -e 's/:$//' -e 's/[^-][^\/]*\//--/g' -e 's/^/ /' -e 's/-/|/'"<br />
<div>
<br /></div>
<div>
Now, when you want to see a complex directory structure in the shell, just change to the working directory, and type `tree`</div>
<div>
<br /></div>
<div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://2.bp.blogspot.com/-X3CypVlVMt4/VvK68v13LdI/AAAAAAAABCk/v1FidRJmhnESJY6H0d-1CycLwWInfLulQ/s1600/tree-example.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="269" src="https://2.bp.blogspot.com/-X3CypVlVMt4/VvK68v13LdI/AAAAAAAABCk/v1FidRJmhnESJY6H0d-1CycLwWInfLulQ/s640/tree-example.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Custom `tree` command</td></tr>
</tbody></table>
<br /></div>
Champion Of Cyrodiilhttp://www.blogger.com/profile/06792737627887393792noreply@blogger.com0tag:blogger.com,1999:blog-6167470039524068257.post-20838924615471106372015-11-15T18:20:00.002-08:002015-11-16T11:19:35.114-08:00Dynamic Adaptive Streaming over HTTP (DASH) on WildFlyI've been away for a while. October 10th, 2015 I started a new project at work, and that has kept me busy.<br />
<br />
As demand has increased on internet media, the core web infrastructure has evolved to support new use cases for some pretty old standards.<br />
<br />
HTTP has been used since the inception of the web around 1989. The core concept being, a client's request receives a peer's response. Originally a content length header element was used to box the request and response body in to some static size. This was used for a variety of reasons, and was also manipulated to perform denial-of-service (DoS) attacks. This is now used to stream partial offsets of a video file, allowing the player to start fetching segments with a lower bit-rate. Hence, dynamic adaptive streaming over http.<br />
<br />
You can check out the wildfly and castlab's code at my github public repository: https://github.com/charlescva/mobile-dashjs<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://1.bp.blogspot.com/-0AYGu3ryKF0/Vkk5L35zaVI/AAAAAAAAA2A/lXnWCFe1wzg/s1600/MobileDashExample.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="433" src="http://1.bp.blogspot.com/-0AYGu3ryKF0/Vkk5L35zaVI/AAAAAAAAA2A/lXnWCFe1wzg/s640/MobileDashExample.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Notice the Content-Length is determined by the offset as provided by the MPD and Initial MP4 containing the Metadata about each stream.</td></tr>
</tbody></table>
<br />
<br />
You can review the source, but the steps are as follows:<br />
<br />
<ol>
<li>Obtain a standard MP4 example video.</li>
<li>Configure Apache to host the files in the directory dashencrypt is using. This is currently hard-coded in the VideoRegistration.java.</li>
<li>Add Video using the Add a Video tab. The JAX-RS enabled VideoService.java will handle the request, and dash the file for you.</li>
<li>Upon success, you will see the entry for the video appear on the "Video Player" tab.</li>
</ol>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://1.bp.blogspot.com/-BfpGJcTco2M/Vkk7k2wSm_I/AAAAAAAAA2M/ZA35p_t45uw/s1600/MobileDashEclipse.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="374" src="http://1.bp.blogspot.com/-BfpGJcTco2M/Vkk7k2wSm_I/AAAAAAAAA2M/ZA35p_t45uw/s640/MobileDashEclipse.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Observing the Console. You can see the logger is outputting the steps as it processes the request.</td></tr>
</tbody></table>
<div>
I am still getting my feet wet as well, and came across a great article on the following website, <a href="https://arashafiei.wordpress.com/2012/11/13/quick-dash/">https://arashafiei.wordpress.com/2012/11/13/quick-dash/</a>. </div>
<div>
<br /></div>
<div>
I'll be working on integrating a "live" stream in which a imaging device like /dev/video0 (webcam) will be used to generate the video segment data, while the MPD (Manifest) and initial MP4 file containing the Movie Box (moov) and/or Fragment Box (moof) are updated on the fly. Essentially, the goal is to enable "DASHing" of a live video feed.</div>
<br />
<br />
<br />Champion Of Cyrodiilhttp://www.blogger.com/profile/06792737627887393792noreply@blogger.com1tag:blogger.com,1999:blog-6167470039524068257.post-16826884152775899032015-08-21T16:17:00.003-07:002015-11-16T11:23:22.366-08:00There was a problem playing this protected content. (Error Code: 3336)<div class="separator" style="clear: both; text-align: center;">
<a href="http://3.bp.blogspot.com/-Q3Sq0nw9kmw/VdethTwCP0I/AAAAAAAAAtY/qyM3yxRE2zg/s1600/hulu-drm-nightmare.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="202" src="http://3.bp.blogspot.com/-Q3Sq0nw9kmw/VdethTwCP0I/AAAAAAAAAtY/qyM3yxRE2zg/s320/hulu-drm-nightmare.png" width="320" /></a></div>
<div style="text-align: center;">
<br /></div>
<br />
Once again, the powers at be, controlling DRM software integration at Google and Hulu have decided to disregard the users of 'free' software and instead prioritize users of Windows, Apple and Google Chrome/Android devices. QoS for the mainstream is understandable. But there is a difference between low quality code that is not efficient or ideal, and code that simply doesn't work.<br />
<br />
What is more annoying, is that these legal issues are masquerading as 'error codes'. It can be determined I'm using linux via browser request headers, and IF the OS is linux, and the error is DRM related, just be honest and tell me that it's not supported on this platform, because of a recent update.<br />
<br />
I'm sure they have been forced to enabled DRM, and waited until the last minute to do so. And didn't have time to rework the baseline framework to support HTML5 for linux users. I'm sure they'll do it eventually, but for now are covering their lack of planning with an error code.<br />
<br />
This is not the first time this has happened. We all remember the days of Linux tweaks getting Netflix to work using VMs, Wine, and god knows what else. Finally, after years of complaining, Netflix allowed HTML5, and all is well... for now.<br />
<br />
It seems that a new update from Hulu and Google does not permit Google Chrome running on Linux variants to support DRM protected content. This is apparently due to licensing issues, copyrights, and cost. It would seem that Hulu blindly added the DRM requirements, without considering the repercussions (Or they just don't care).<br />
<br />
Not surprising, small businesses with a great idea like Hulu once was, has become a draconian enterprise in which legal pressure and money woes force a quality product to become sub-par and only available to paid platforms.<br />
<br />
After working a decade in software engineering, I have realized that what is important to those of us creating the software for the rest of society, is not relevant to the mainstream. The same users who would tell you Linux is 'shit' because you can't be productive, are the ones who will ultimately whine and complain like a child whenever their Windows boot partition shits the bed.<br />
<br />
As expected, the Linux base is now responding with the usual rhetoric about leaving Hulu and not giving them any more money. There are some workarounds being expressed. But likely I'll switch to either my Windows box, or I'll end up going w/ Firefox + HAL.<br />
<br />
For now, here are the most informative forums with users doing exactly what I described above.<br />
<br />
<ul>
<li><a href="https://productforums.google.com/forum/#!topic/chrome/H_WqDsZFS9w">https://productforums.google.com/forum/#!topic/chrome/H_WqDsZFS9w</a></li>
<li><a href="http://ubuntuforums.org/showthread.php?t=2290743">http://ubuntuforums.org/showthread.php?t=2290743</a> (Marked "Solved", even though it isnt)</li>
<li><a href="http://www.linuxquestions.org/questions/linux-software-2/note-hulu-playback-in-chrome-no-longer-working-4175550849/">http://www.linuxquestions.org/questions/linux-software-2/note-hulu-playback-in-chrome-no-longer-working-4175550849/</a></li>
</ul>
<div>
Looking forward to the comments from those of you who 'know' whats really going on.</div>
Champion Of Cyrodiilhttp://www.blogger.com/profile/06792737627887393792noreply@blogger.com1tag:blogger.com,1999:blog-6167470039524068257.post-39581162444633139432015-08-20T14:24:00.001-07:002015-08-20T14:36:14.184-07:00Collecting Apache Storm time series data with GraphiteNaturally, Apache Storm processes data as tuples over time. This is an ideal framework to utilize for streaming data through a 'pipe line'. However, maintaining a record of this information is not necessarily an inherent feature of storm. For metrics collection and analysis, I used Graphite. Graphite is available in the Fedora EPEL and a great option for quickly collecting some metrics and generating a graph w/ overlaying analytical functions.<br />
<br />
The easiest way to collect information is by going straight to the source. The Nimbus server. Using the <a href="http://storm.apache.org/apidocs/backtype/storm/utils/NimbusClient.html">NimbusClient</a> class, available from the package <i>backtype.storm.generated</i>, you can easily review the <a href="http://storm.apache.org/apidocs/backtype/storm/generated/ClusterSummary.html">ClusterSummary</a> and iterate through each <a href="http://storm.apache.org/apidocs/backtype/storm/generated/TopologySummary.html">TopologySummary</a>'s ID to retrieve <a href="http://storm.apache.org/apidocs/backtype/storm/generated/TopologyInfo.html">TopologyInfo</a> and each topology's <a href="http://storm.apache.org/apidocs/backtype/storm/generated/ExecutorSummary.html">ExecutorSummary</a>, to determine emitted and transferred tuples. Additionally, you can get information about errors, threads, and likely other details related to Apache Storm topologies I have not collected in my example code.<br />
<br />
First, Lets look at how I will push data in to the carbon-cache daemon.<br />
<a href="https://github.com/charlescva/graphite-common/blob/master/src/main/java/zkCliTest.java#L44">https://github.com/charlescva/graphite-common/blob/master/src/main/java/zkCliTest.java#L44</a><br />
<br />
<blockquote class="tr_bq">
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;"> // TCP Stream to carbon-cache.<br /> private void graphite(Map<string string=""> metrics) {<br /> // Current Time-Stamp for test<br /> long epoch = System.currentTimeMillis()/1000;<br /> <br /> try { // output stream to the host on default port.<br /> Socket conn = new Socket("cabon-cache.novalocal", 2003);<br /> DataOutputStream dos = new DataOutputStream(conn.getOutputStream());<br /> // graphite syntax map the #ngsm to output stream.<br /> for (String metric : metrics.keySet() ){<br /> dos.writeBytes(metric + <br /> " " + metrics.get(metric) + <br /> " " + epoch + "\n");<br /> }<br /> //CLOSED CONNECTION.<br /> conn.close();<br /> } catch (IOException e) {<br /> e.printStackTrace();<br /> }<br /> }</string></span></span></blockquote>
The above code simply opens a TCP socket to the carbon-cache, writes a Map of metrics with the current timestamp, and closes the connection. Easy. The Take-away here, is the syntax. '[metric_path] [value] [time]' as String.<br />
<br />
It would be redundant for me to display all of the Java code here: <a href="https://github.com/charlescva/graphite-common/blob/master/src/main/java/zkCliTest.java#L125">https://github.com/charlescva/graphite-common/blob/master/src/main/java/zkCliTest.java#L125</a> which essentially takes a Nimbus client connect, and builds a model of the nimbus state with your topoloy as each child node of the root context.<br />
<br />
So, rather than bore you. Here is a screenshot!<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://3.bp.blogspot.com/-bWnzysoT9h8/VdZFQpanVyI/AAAAAAAAAtE/MK58NukVq-8/s1600/Screenshot%2Bfrom%2B2015-08-20%2B17%253A18%253A13.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="304" src="http://3.bp.blogspot.com/-bWnzysoT9h8/VdZFQpanVyI/AAAAAAAAAtE/MK58NukVq-8/s640/Screenshot%2Bfrom%2B2015-08-20%2B17%253A18%253A13.png" width="640" /></a></div>
<br />
As you can see, the "My Topology" is a test, and clearly a static source that is linear. But all and all, you can quickly get some good information. Feel free to comment, as I find this article particularly interesting.Champion Of Cyrodiilhttp://www.blogger.com/profile/06792737627887393792noreply@blogger.com1tag:blogger.com,1999:blog-6167470039524068257.post-17132366722780047052015-08-06T01:56:00.000-07:002015-08-20T14:25:54.786-07:00Intel NUC won't turn on (after Ubuntu suspend)<div class="tr_bq">
UPDATE: THE SOLUTION TO THIS PROBLEM IS IN THE COMMENTS! READ ON IF YOU WANT TO MAKE SURE YOUR PROBLEM REQUIRES THE SOLUTION! <br />
<br />
Just got my NUC back up and running before posting this blog. It doesn't surprise me much, as we have had a lot of issues with the Core i3 model at work. I will say though, after ordering around 30 of these things, we have not had a single one brick or be dead on arrival. Anyway, back to this issue.</div>
<br />
I have been using a Microsoft Remote to power my NUC on and off via the Infrared sensor on the front. I had to install a custom kernel module to get the operating system to understand the signal correctly. As a result, I noticed if I use the remote to suspend the unit, <b>I have to use the remote to turn it back on.</b> Pressing the soft switch on the unit itself does nothing. I'm not sure if this issue is w/ the linux community drivers, intel, or both, but it is likely to be related.<br />
<br />
The issue seems to be with the way the NUC hibernates after a software power down signal. The BIOS/CMOS (whatever it's called these days), seems to store a bit somewhere in memory along with the system clock information related to the power state. So <b>the only way to get my NUC to power back on was to remove the battery from the motherboard, wait about five minutes for good measure, r</b><b>eplace it,</b> put the NUC back together, and boot on up.<br />
<br />
To get to the bottom of the motherboard, you should first remove RAM and SSD (unless you really don't care about them.) I was able to keep wifi connectors attached, but be gentle not to yank them off. There are two black phillips head (+) screws located on the sides. Remove these in front of a cat so he/she will look directly at where they land when you drop them on the floor.<br />
<br />
<b>Tricky Part:</b> Turn the NUC upside down w/ your hand on the bottom to prevent the board from falling out and gently pry the back out w/ your finger. Once the back is lifted up, you can shift/slide the board 1/16 of an inch (or about 1.5 cm) towards the rear to get the headphone jack and IR sensor clear of the holes in the chassis.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://4.bp.blogspot.com/-_5g3Y1hHuWg/VcMbsa8c95I/AAAAAAAAAsE/zbEz98gSI_4/s1600/IMG_20150806_040830_631.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><br class="Apple-interchange-newline" /><img border="0" height="320" src="http://4.bp.blogspot.com/-_5g3Y1hHuWg/VcMbsa8c95I/AAAAAAAAAsE/zbEz98gSI_4/s320/IMG_20150806_040830_631.jpg" width="180" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<i><span style="font-size: x-small;">Battery is located to the left and removed via 2-pin connector</span></i></div>
<div>
<i><span style="font-size: x-small;"><br /></span></i></div>
At this point the motherboard should be out and you can flip it around and get to the battery.<br />
<br />
Also interesting is that on first boot, my mSATA SSD could not be detected. Of course this initially had me worried, but after a second reboot, the OS discovered the disk just fine.<br />
<br />
Not sure if a firmware update would have fixed this, but I've been running this NUC for more than 6 months without issue and am not going to bother w/ a firmware update at this time.<br />
<br />
Likely related is the kernel modules used to power on/off the NUC via remote. Here are the ones I have loaded:<br />
<blockquote class="tr_bq">
<br />
<i>charlie@NUC:~$ sudo lsmod | grep -i IR</i><i>ir_sony_decoder 12713 0 </i><i>ir_lirc_codec 13021 0 </i><i>lirc_dev 19980 1 ir_lirc_codec</i><i>ir_mce_kbd_decoder 13214 0 </i><i>ir_sanyo_decoder 12839 0 </i><i>ir_jvc_decoder 12751 0 </i><i>ir_rc6_decoder 12874 0 </i><i>ir_rc5_decoder 12710 0 </i><i>ir_nec_decoder 12915 0 </i><i>nuvoton_cir 17778 0 </i><i><b>rc_core 28124 12 lirc_dev,nuvoton_cir,ir_lirc_codec,ir_rc5_decoder,ir_nec_decoder,ir_sony_decoder,ir_mce_kbd_decoder,ir_jvc_decoder,ir_rc6_decoder,ir_sanyo_decoder,rc_rc6_mce</b></i></blockquote>
<div>
<br /></div>
And some additional information about my model/firmware.<br />
<blockquote>
<i><br />BIOS Information<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>Vendor: Intel Corp.<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>Version: WYLPT10H.86A.0038.2015.0410.1634<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>Release Date: 04/10/2015<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>Address: 0xF0000<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>Runtime Size: 64 kB<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>ROM Size: 6656 kB<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>Characteristics:<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>PCI is supported<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>BIOS is upgradeable<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>BIOS shadowing is allowed<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>Boot from CD is supported<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>Selectable boot is supported<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>BIOS ROM is socketed<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>EDD is supported<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>5.25"/1.2 MB floppy services are supported (int 13h)<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>3.5"/720 kB floppy services are supported (int 13h)<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>3.5"/2.88 MB floppy services are supported (int 13h)<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>Print screen service is supported (int 5h)<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>Serial services are supported (int 14h)<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>Printer services are supported (int 17h)<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>ACPI is supported<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>USB legacy is supported<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>BIOS boot specification is supported<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>Targeted content distribution is supported<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>UEFI is supported<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>BIOS Revision: 4.6<br />Base Board Information<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>Manufacturer: Intel Corporation<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>Product Name: D34010WYK<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>Version: H14771-303<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>Serial Number: XXXXXXXXXXXX (redacted)<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>Asset Tag: <br /><span class="Apple-tab-span" style="white-space: pre;"> </span>Features:<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>Board is a hosting board<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>Board is replaceable<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>Location In Chassis: To be filled by O.E.M.<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>Chassis Handle: 0x0003<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>Type: Motherboard<br /><br />Ubuntu 14.04.2 (amd64)<br />Linux Kernel 3.13.0-61</i></blockquote>
Champion Of Cyrodiilhttp://www.blogger.com/profile/06792737627887393792noreply@blogger.com6tag:blogger.com,1999:blog-6167470039524068257.post-19908406458352440642015-06-04T10:53:00.001-07:002015-06-04T11:05:09.252-07:00Openstack DHCP Lease renewing every minute on all VMs?!Been having performance issues w/ the Openstack cluster. User's have mentioned that occasionally VMs are unresponsive.<br />
<br />
I came across this article describing some of the common issues w/ DHCP, <a href="https://www.mirantis.com/blog/improving-dhcp-performance-openstack/">Improving DHCP Performance in OpenStack</a>.<br />
<br />
The article explains that the default lease duration is 120 seconds. In order to ensure dhcp clients (VMs) don't lose their IP, the client daemon attempts a renewal halfway through the lease duration. Which is approximately +/-60 seconds.<br />
<br />
In my case, I have about 20 virtual machines hitting a single DHCP agent every 60 seconds.<br />
<br />
VMs on an Openstack network (private) will not be accessible (from public) during this time, causing periods of 'lock up' for users.<br />
<br />
The fix is to simply increase DHCP lease duration to something a bit more sane for your platforms.<br />
In my case, I chose 600 seconds, thus givng a nice 5 minute window between lease renewals.<br />
<br />
However, if running Juno with HA, pacemaker will maintain a running dhcp agent on any host in your cluster. This can be confusing since you will see the agent running on a node, even though the service is 'stopped'. <br />
<br />
The fix is just to update the config on your nodes, and then disable/enable the resource via pacemaker.<br />
<br />
<br />
# For each node in your cluster, from a controller as root run:<br />
<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">[root@node-54 ~]# ssh node-46 -C "echo dhcp_lease_duration = 600 >> /etc/neutron/dhcp_agent.ini"</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">[root@node-54 ~]# ssh node-47 -C "echo dhcp_lease_duration = 600 >> /etc/neutron/dhcp_agent.ini"</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">[root@node-54 ~]# ssh node-50 -C "echo dhcp_lease_duration = 600 >> /etc/neutron/dhcp_agent.ini"</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">[root@node-54 ~]# ssh node-51 -C "echo dhcp_lease_duration = 600 >> /etc/neutron/dhcp_agent.ini"</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">[root@node-54 ~]# ssh node-52 -C "echo dhcp_lease_duration = 600 >> /etc/neutron/dhcp_agent.ini"</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">[root@node-54 ~]# ssh node-53 -C "echo dhcp_lease_duration = 600 >> /etc/neutron/dhcp_agent.ini"</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">[root@node-54 ~]# ssh node-54 -C "echo dhcp_lease_duration = 600 >> /etc/neutron/dhcp_agent.ini"</span><br />
<br />
<div>
Next, disable and then enable the service:</div>
<div>
<br /></div>
<div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">[root@node-54 ~]# pcs resource disable p_neutron-dhcp-agent</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">[root@node-54 ~]# pcs resource enable p_neutron-dhcp-agent</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">[root@node-54 ~]# neutron agent-list</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">+--------------------------------------+--------------------+------------------+-------+----------------+---------------------------+</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">| id | agent_type | host | alive | admin_state_up | binary |</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">+--------------------------------------+--------------------+------------------+-------+----------------+---------------------------+</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">| 11a5d775-7f10-4856-b905-a6176e9cb689 | Open vSwitch agent | node-46.example.com | :-) | True | neutron-openvswitch-agent |</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">| 14b47084-d1f5-428c-8fed-746a5d5e743f | Metadata agent | node-54.</span><span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">example</span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;">.com | :-) | True | neutron-metadata-agent |</span></div>
<div>
<span style="color: lime;"><span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><b>| 311fe911-503d-498f-a027-72414c6c8664 | DHCP agent | node-46.</b></span><span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">example</span></span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><b><span style="color: lime;">.com | :-) | True | neutron-dhcp-agent </span> |</b></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">| 326c7dcc-03b0-4830-8a4e-6b58f3af2445 | Open vSwitch agent | node-51.</span><span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">example</span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;">.com | :-) | True | neutron-openvswitch-agent |</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">| 3867b579-1bc2-4aee-b9b6-5bc21f61f514 | Open vSwitch agent | node-54.</span><span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">example</span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;">.com | :-) | True | neutron-openvswitch-agent |</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">| 3de48cde-8dba-4042-9a04-baa9f3d411f9 | Open vSwitch agent | node-50.</span><span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">example</span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;">.com | :-) | True | neutron-openvswitch-agent |</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">| 5b85f41e-2671-4bf6-8dc5-243a0ecb55b3 | L3 agent | node-50.</span><span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">example</span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;">.com | :-) | True | neutron-l3-agent |</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">| 61b3c026-99e0-4c50-a440-339b7085d428 | L3 agent | node-46.</span><span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">example</span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;">.com | :-) | True | neutron-l3-agent |</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">| 6e9b071d-5417-45a2-abf6-72b2691fd464 | Open vSwitch agent | node-47.</span><span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">example</span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;">.com | :-) | True | neutron-openvswitch-agent |</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">| 91e124a6-6594-4fb4-b48c-06b697cbf437 | L3 agent | node-54.</span><span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">example</span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;">.com | :-) | True | neutron-l3-agent |</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">| 9881ddce-f177-4120-9019-1fc26eee19ca | Open vSwitch agent | node-52.</span><span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">example</span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;">.com | :-) | True | neutron-openvswitch-agent |</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">| 9e259924-80ce-44c8-9704-8cb8ed35d751 | Metadata agent | node-50.</span><span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">example</span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;">.com | :-) | True | neutron-metadata-agent |</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">| c5b75fd5-f83b-4afb-a4ff-271c92d61695 | Open vSwitch agent | node-53.</span><span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">example</span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;">.com | :-) | True | neutron-openvswitch-agent |</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">| cd99597e-ead6-4eb3-a729-4bc609955ee6 | Metadata agent | node-46.</span><span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">example</span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;">.com | :-) | True | neutron-metadata-agent |</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">+--------------------------------------+--------------------+------------------+-------+----------------+---------------------------+</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">[root@node-54 ~]# neutron agent-show <b>311fe911-503d-498f-a027-72414c6c8664</b></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">+---------------------+---------------------------------------------------------+</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">| Field | Value |</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">+---------------------+---------------------------------------------------------+</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">| admin_state_up | True |</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">| agent_type | DHCP agent |</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">| alive | True |</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">| binary | neutron-dhcp-agent |</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">| configurations | { |</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">| | "subnets": 5, |</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">| | "use_namespaces": true, |</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">| | <b><span style="color: lime;">"dhcp_lease_duration": 600,</span></b> |</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">| | "dhcp_driver": "neutron.agent.linux.dhcp.Dnsmasq", |</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">| | "networks": 5, |</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">| | "ports": 39 |</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">| | } |</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">| created_at | 2015-06-04 17:35:57 |</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">| description | |</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">| heartbeat_timestamp | 2015-06-04 17:36:46 |</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">| host | node-46.ccri.com |</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">| id | 311fe911-503d-498f-a027-72414c6c8664 |</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">| started_at | 2015-06-04 17:35:57 |</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">| topic | dhcp_agent |</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">+---------------------+---------------------------------------------------------+</span></div>
</div>
<div>
<br />
<br />
The result is that the client will repeat DHCPREQUEST until the service is back up, and start using the new lease time,<br />
<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">root@hannibal:~# tail -f /var/log/syslog</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">Jun 4 17:35:02 hannibal dhclient: DHCPACK of 192.168.111.95 from 192.168.111.2</span><br />
<span style="color: lime; font-family: Courier New, Courier, monospace; font-size: x-small;"><b>Jun 4 17:35:02 hannibal dhclient: bound to 192.168.111.95 -- renewal in 51 seconds.</b></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">Jun 4 17:35:53 hannibal dhclient: DHCPREQUEST of 192.168.111.95 on eth0 to 192.168.111.2 port 67 (xid=0x50f4018e)</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">Jun 4 17:36:35 hannibal dhclient: message repeated 6 times: [ DHCPREQUEST of 192.168.111.95 on eth0 to 192.168.111.2 port 67 (xid=0x50f4018e)]</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">Jun 4 17:36:37 hannibal dhclient: DHCPNAK from 192.168.111.2 (xid=0x50f4018e)</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">Jun 4 17:36:37 hannibal dhclient: DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 3 (xid=0x75f1e5db)</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">Jun 4 17:36:37 hannibal dhclient: DHCPREQUEST of 192.168.111.95 on eth0 to 255.255.255.255 port 67 (xid=0x75f1e5db)</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">Jun 4 17:36:37 hannibal dhclient: DHCPOFFER of 192.168.111.95 from 192.168.111.2</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">Jun 4 17:36:37 hannibal dhclient: DHCPACK of 192.168.111.95 from 192.168.111.2</span><br />
<span style="color: lime; font-family: Courier New, Courier, monospace; font-size: x-small;"><b>Jun 4 17:36:37 hannibal dhclient: bound to 192.168.111.95 -- renewal in 255 seconds.</b></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">Jun 4 17:40:52 hannibal dhclient: DHCPREQUEST of 192.168.111.95 on eth0 to 192.168.111.2 port 67 (xid=0x75f1e5db)</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">Jun 4 17:40:52 hannibal dhclient: DHCPACK of 192.168.111.95 from 192.168.111.2</span><br />
<span style="color: lime; font-family: Courier New, Courier, monospace; font-size: x-small;"><b>Jun 4 17:40:52 hannibal dhclient: bound to 192.168.111.95 -- renewal in 211 seconds.</b></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">Jun 4 17:44:23 hannibal dhclient: DHCPREQUEST of 192.168.111.95 on eth0 to 192.168.111.2 port 67 (xid=0x75f1e5db)</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">Jun 4 17:44:23 hannibal dhclient: DHCPACK of 192.168.111.95 from 192.168.111.2</span><br />
<span style="color: lime; font-family: Courier New, Courier, monospace; font-size: x-small;"><b>Jun 4 17:44:23 hannibal dhclient: bound to 192.168.111.95 -- renewal in 262 seconds.</b></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">Jun 4 17:45:01 hannibal CRON[4249]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">Jun 4 17:48:45 hannibal dhclient: DHCPREQUEST of 192.168.111.95 on eth0 to 192.168.111.2 port 67 (xid=0x75f1e5db)</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">Jun 4 17:48:45 hannibal dhclient: DHCPACK of 192.168.111.95 from 192.168.111.2</span><br />
<span style="color: lime; font-family: Courier New, Courier, monospace; font-size: x-small;"><b>Jun 4 17:48:45 hannibal dhclient: bound to 192.168.111.95 -- renewal in 249 seconds.</b></span></div>
<div>
<br />
<br />
<span style="font-size: large;">IMPORTANT!!!</span><br />
<span style="font-family: inherit;"><i>"Attempting to work around these performance problems by significantly increasing IP lease time will cause a huge problem with respect to the release of IP addresses by neutron if your cloud loads dynamically change. By default, neutron will allocate an IP address to a VM for 24 hours, independent of the actual lease time. Also, by default, neutron will not release an IP address until 24 hours after an instance has been terminated."</i></span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;"></span><br />
<span style="font-family: inherit;"> - <a href="https://www.mirantis.com/blog/improving-dhcp-performance-openstack/">https://www.mirantis.com/blog/improving-dhcp-performance-openstack/</a></span></div>
Champion Of Cyrodiilhttp://www.blogger.com/profile/06792737627887393792noreply@blogger.com1tag:blogger.com,1999:blog-6167470039524068257.post-40469753079182581332015-02-26T19:14:00.001-08:002015-03-02T06:02:17.587-08:00Recover Openstack Ceph data with missing/no monitor(s)Recently ceph monitors got blown away. Along with this was all of the metadata associated with the monitors. Using this technique I was able to recover some of my data, but it was a lot of sleuthing<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://2.bp.blogspot.com/-NFfqC01h7KA/VO_bevTcQbI/AAAAAAAAAlY/l8JzExZLPPU/s1600/ceph-recovery.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://2.bp.blogspot.com/-NFfqC01h7KA/VO_bevTcQbI/AAAAAAAAAlY/l8JzExZLPPU/s1600/ceph-recovery.png" height="368" width="640" /></a></div>
<br />
<br />
In the top left corner are the script running in a loop over all of the unique 'header' files from the various osds.<br />
<br />
The main script is in the top right corner. Essentially we traverse the servers (nodes) and ceph osd instances throughout the cluster, collecting files (with find) that match the wildcard and are bigger than a byte.<br />
<br />
The "wildcard" is the key, "13f2a30976b17" which is defined as replicated header file names for each rbd image on your ceph cluster. If you had 10 images, with 3 replicas, you would find 30 header files in your cluster, with identical names for the replicas. This would be okay, even if they are on the same server; because they are in separate osd data folders.<br />
<br />
Using SSH we fetch a list of all the files on an osd instance and dump to a temp file. We do a cut on the slash(/) folder separator and dump a list of the files in a new file and remove the temp.<br />
<br />
We then dump all the files into a csv, with the osd node location in column 1 and the file name in column 2. the -u switch only snags unique instances, so replicas are dropped.<br />
<br />
We then execute a little script called scp obs. the tricky part here is the backslash in the ceph file names. use double quotes in the scp command and escape the \ with \\. So that's 3 slashes surrounded in double quotes w/ the scp command. <br />
<br />
finally once we have all the object files. we 'dd' them together ans the final output.<br />
<br />
Two quick notes,<br />
<br />
in my cut command i use column #8 and #9. Thinking about it, this could give you a different result depending on where your osd data folder is. Mine is the default path, /var/lib/ceph/osd/ceph-0/current/4.4f/<obs and="" headers="" on="" osd="" the=""></obs><br />
<br />
For my convenience, at the end I mv the "raw" file to qcow2, since I know that is what these images are. This is based on the output of hexdump -C -n 4 -s 0 $first-block, where the first-block is the object with 16 zeroes. (the first block in the object group). It basically tells me the header of the first block which is 'QFI' for qcow2.<br />
<br />
I even converted one of the qcow2 files to a VDI and booted in successfully in virtualbox.<br />
<br />
<b><span style="font-size: large;">The bash scripts can be found here:</span></b><br />
<a href="https://github.com/charlescva/ceph-recovery"><b><span style="font-size: large;">https://github.com/charlescva/ceph-recovery</span></b></a><br />
<br />
UPDATE:<br />
It is the next morning, and I let my script run overnight. Check it out. :)<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://4.bp.blogspot.com/-mECQkNYiYi0/VPBxfVZdxFI/AAAAAAAAAlo/7mfA56r3Ihc/s1600/Screenshot%2Bfrom%2B2015-02-27%2B08%3A29%3A41.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://4.bp.blogspot.com/-mECQkNYiYi0/VPBxfVZdxFI/AAAAAAAAAlo/7mfA56r3Ihc/s1600/Screenshot%2Bfrom%2B2015-02-27%2B08%3A29%3A41.png" height="554" width="640" /></a></div>
<br />Champion Of Cyrodiilhttp://www.blogger.com/profile/06792737627887393792noreply@blogger.com4tag:blogger.com,1999:blog-6167470039524068257.post-82217595434235095872015-02-18T20:33:00.000-08:002015-02-18T20:59:38.205-08:00Administering Fuel with Openstack Juno ServicesI have recently started using Openstack in an environment with 'production' value. By this I mean that our Openstack instance is becoming a critical component of our business infrastructure; and at this point several development support services are tenants within it.<br />
<br />
Openstack is not an easy solution. Almost every core service is distributed, decentralized, and utilizes the full scope of their dependencies. This results in good news, and bad news. The good news is that your infrastructure is so loosely coupled, that failures will USUALLY be localized to a specific process or configuration setting. The bad news is, until you learn the terminology and components, you'll be running around like a mad man trying to find the various configs and error logs.<br />
<br />
<b>Ceph</b><br />
<br />
First you will need to ensure your file system is stable. Ceph has been with Openstack since for a long time. Yes it is different than any other file system you're likely used to. This means you'll have to learn something new. One of the biggest issues with migration and spawning VMs can stem from failures to read/write RAW data to the distributed file system.<br />
<br />
The best thing to do first, is read over this paper on RUSH, or replication under scalable hashing: <a href="http://www.ssrc.ucsc.edu/Papers/honicky-ipdps04.pdf">http://www.ssrc.ucsc.edu/Papers/honicky-ipdps04.pdf</a>.<br />
<br />
The gist of this paper should help you to understand that Ceph clients in Openstack use the jenkins hash (<a href="http://en.wikipedia.org/wiki/Jenkins_hash_function)">http://en.wikipedia.org/wiki/Jenkins_hash_function)</a> with a tree of weighted buckets (CRUSH Map, <a href="http://ceph.com/docs/master/rados/operations/crush-map/">http://ceph.com/docs/master/rados/operations/crush-map/</a>) and a map defaulting of 256 placement groups (<a href="http://ceph.com/docs/master/rados/operations/placement-groups/">http://ceph.com/docs/master/rados/operations/placement-groups/</a>) to figure out where objects are stored. Also that Ceph is not a file system, per say, but an "object store". This means there is no central server the clients must negotiate with to read and write object data. The Ceph documentation is phenomenal, and you should familiarize yourself with it is much as you can. Most of your questions are answered in the documentation, you'll just need to be patient, read it all at a decent pace, and let the information resonate with your mind for a night before digging in to it again. After a couple of days it will start to make more sense. Here are some common commands to take a peak at:<br />
<br />
<ul>
<li>ceph osd tree</li>
<li>ceph -w</li>
<li>ceph osd reweight (don't just run this randomly, understand what it does first)</li>
</ul>
<div>
Also keep in mind there have been bug reports regarding applying a new Crush map to a running cluster. So spend a lot of time looking at a sample crush map in a test cluster before applying a new one. It is likely that you can resolve a lot of your issues by using reweight and or modifying the number of replicas in largely used storage pools. like your Openstack volumes, images and compute pool for ephemeral storage</div>
<div>
<br /></div>
<div>
<b>RBD (Rados Block Device)</b></div>
<div>
<b><br /></b></div>
<div>
RBD is used on top of the Ceph object store. This provides the API Openstack uses to connect your volumes and images to the hypervisor you're using (Hopefully QEMU, because I like it and want it supported). Here are some helpful commands:</div>
<div>
<ul>
<li>rados df</li>
<li>rbd import</li>
<li>rbd export</li>
<li>rbd ls|rm|mv</li>
<li>qemu-img convert (although not rbd specific, relvent when dealing with RAW rbd images and compressing them to qcow2 for moving across the network)</li>
</ul>
<div>
In an earlier post on this blog, you will see my experience upgrading openstack. In there you will see where I manually migrated each of my VMs from an Icehouse cluster to Juno. I had some hardware constraints and it was tough, but in the end it worked very well. </div>
</div>
<div>
<br /></div>
<div>
<b>nova,cinder,glance CLI</b></div>
<div>
<br /></div>
<div>
You won't get by on the UI alone. The bash command line for an openstack controller is your best tool. Don't be afraid to poke around the databases on mysql for cinder, glance and nova. Use the nova, glance and cinder tools with the 'help' argument and read the usage. These tools are required to communicate with the API in a standardized way that is supported by the developers of Openstack. If you're using 3rd party providers like Mirantis Fuel for Openstack, then you will need to use their documentation for maintaining Openstack environments. Be advised, some of these 3rd party tools are lacking support and capability to perform some of the tasks you will need to know to properly maintain the environment.</div>
<div>
<br /></div>
<div>
Here are the ones to know:</div>
<div>
<ul>
<li>nova boot</li>
<ul>
<li>--availability-zone</li>
<li>--nic id</li>
<li>--flavor</li>
<li>flags for Volume or Image backed.</li>
</ul>
<li>nova services-list</li>
<li>nova service-delete (Gets mention for not in Havana, but is in Juno!)</li>
</ul>
<div>
Seriously though, use mysql and don't be affraid to adjust the instances metadata. Sometimes a VM is actually OFF, but the Horizon UI will show it as 'Shutting Down...' or 'Running'. You can verify the status of your VM by SSHing into the compute node hosting the instance, and as root running:</div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"># ps -ef | grep kvm</span></div>
<div>
<br /></div>
<div>
You'll see the instance id in the run command, as well as a bunch of other args. Be advised, the domain.xml virsh uses is generated in code by python and uses the information in mysql to do so. So modifying things like the video driver or video ram, require changes to the flavor and image metadata. I recently saw in Juno an option to nova boot with args passing metadata key values to set in the virsh domain, although I have not tried it yet. I believe it is here: <a href="http://docs.openstack.org/cli-reference/content/novaclient_commands.html#novaclient_subcommand_boot">http://docs.openstack.org/cli-reference/content/novaclient_commands.html#novaclient_subcommand_boot</a>, and the boot option appears to be --image-width <key value="">.</key></div>
<div>
<br /></div>
<div>
<b>Neutron</b></div>
</div>
<div>
<b><br /></b></div>
<div>
Neutron is a bit overwhelming. Just know that the Open vSwitch service on your compute nodes handle the networking for the VMs running there. Just because your L3 Agent(s) are down and you cannot get to the VM using it's public IP, does not mean that the VM is Off, it just means that the external connection isnt being routed. Ensure all of these services are running and configured correctly. This section is intentionally short because of the vast configuration options with neutron.</div>
<div>
<ul>
<li>neutron list-agents</li>
</ul>
<div>
Last I need to thank the developers at Mirantis Fuel and others hanging out on the freenode IRC channel #fuel. I could not have learned as much as I know at this point, without the help of a few users in there. Thank you guys for your gracious support throughout my adoption of Openstack.</div>
</div>
Champion Of Cyrodiilhttp://www.blogger.com/profile/06792737627887393792noreply@blogger.com0tag:blogger.com,1999:blog-6167470039524068257.post-32863144702475202222015-02-02T14:58:00.001-08:002015-02-03T17:42:49.963-08:00maven: Target server failed to respondWas getting an exception with the maven (3.2.2) wagon plugin version 2.6.<br />
<br />
Looking deeper, this plugin uses wagon-provider 2.6 which depends on httpclient 4.3. This can be seen here: <a href="http://repo1.maven.org/maven2/org/apache/maven/wagon/wagon-providers/2.6/wagon-providers-2.6.pom">http://repo1.maven.org/maven2/org/apache/maven/wagon/wagon-providers/2.6/wagon-providers-2.6.pom</a><br />
and looks like this:<br />
<br />
<style type="text/css">
pre.CICodeFormatter{
font-family:arial;
font-size:12px;
border:1px dashed #CCCCCC;
width:99%;
height:auto;
overflow:auto;
background:#f0f0f0;
line-height:20px;
padding:0px;
color:#000000;
text-align:left;
}
pre.CICodeFormatter code{
color:#000000;
word-wrap:normal;
}
</style>
<br />
<pre class="CICodeFormatter"><code class="CICodeFormatter"> ...
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
<version>4.3.1</version>
</dependency> ...
</code></pre>
<br />
The exception seen when running a maven build looks similar to this:<br />
<br />
<b>Caused by: org.apache.maven.wagon.providers.http.httpclient.NoHttpResponseException: The target server failed to respond</b><br />
<br />
This bug: <a href="https://issues.apache.org/jira/browse/HTTPCLIENT-1531">https://issues.apache.org/jira/browse/HTTPCLIENT-1531</a><br />
<br />
indicates
that there is a bug with httpclient in the versions 4.3 >= 4.3.4
& 4.4 Alpha1 when using a proxy (like apache httpd) between the
client and server, without client authentication enabled. The bug
causes the no response exception later in the wagon plugin called via
the WagonRepositoryConnector class.<br />
<br />
Specifically,
the bug occurs when authentication is disabled because as of 4.3.x the
MainClientExec.java has a function to create tunnel to target,
(createTunnelToTarget) that when authentication is disabled, the for
loop does not exit properly and the request is never completed.<br />
<br />
<a href="http://svn.apache.org/repos/asf/httpcomponents/httpclient/branches/4.3.x/httpclient/src/main/java/org/apache/http/impl/execchain/MainClientExec.java">http://svn.apache.org/repos/asf/httpcomponents/httpclient/branches/4.3.x/httpclient/src/main/java/org/apache/http/impl/execchain/MainClientExec.java</a><br />
<br />
<br />
In
our case, just upgrading to 3.2.5 worked. Another option is to
include the lightweight http provider with wagon configuration. this
uses the Java HTTP libraries instead of apache's implementation.Champion Of Cyrodiilhttp://www.blogger.com/profile/06792737627887393792noreply@blogger.com0tag:blogger.com,1999:blog-6167470039524068257.post-58498591253897267342015-01-27T06:39:00.000-08:002015-01-27T06:52:45.773-08:00How to configure Dell BMC<span style="font-family: Arial, Helvetica, sans-serif; font-size: x-small;">When configuring a Dell PowerEdge C6220 (Or Similar) to be remotely administrated with BMC you will need to modify the BIOS configuration. In this example, a dedicated NIC is used.
</span><br />
<span style="font-family: Arial, Helvetica, sans-serif; font-size: x-small;">Steps:
</span><br />
<ol 0px="" 10px="" margin:="">
<li><span style="font-family: Arial, Helvetica, sans-serif; font-size: x-small;">Boot the Server and Press F2 to enter the BIOS.</span></li>
<li><span style="font-family: Arial, Helvetica, sans-serif; font-size: x-small;">From the BIOS, use the right arrow key to navigate to the "Server" menu, then move down to the BMC Administration.</span></li>
<li><span style="font-family: Arial, Helvetica, sans-serif; font-size: x-small;">Configure the IP Address for the BMC and set the Inteface from "Shared-NIC" to "Dedicated NIC".</span></li>
<li><span style="font-family: Arial, Helvetica, sans-serif; font-size: x-small;">Press Esc to get back to the main menu, then use the right arrow key and navigate all the way to last menu and Save the configuration. (Do not exit the BIOS at this time.)</span></li>
<li><span style="font-family: Arial, Helvetica, sans-serif; font-size: x-small;">Connect an Ethernet cable to the Dedicated BMC port (Identified with a open ended wrench icon) and plug the other end into your network LAN switch.</span></li>
<li><span style="font-family: Arial, Helvetica, sans-serif; font-size: x-small;">From another PC, use a web browser to connect to http://<bmc_ip_addr> and use the username root/root to log in.</bmc_ip_addr></span></li>
</ol>
<div 0px="" 10px="" margin-top:="" padding:="">
<div class="separator" style="clear: both; text-align: center;">
<a href="http://4.bp.blogspot.com/-WB_sp9faxuE/VMelj1Mx6rI/AAAAAAAAAjw/XmG4INUXydA/s1600/Dell%2BRemote%2BManagement%2BConsole.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://4.bp.blogspot.com/-WB_sp9faxuE/VMelj1Mx6rI/AAAAAAAAAjw/XmG4INUXydA/s1600/Dell%2BRemote%2BManagement%2BConsole.png" height="371" width="640" /></a></div>
<div style="text-align: center;">
<span style="font-family: Arial, Helvetica, sans-serif; font-size: x-small;"><br /></span></div>
<span style="font-family: Arial, Helvetica, sans-serif; font-size: x-small;"><br /></span></div>
<div 0px="" 10px="" margin-top:="" padding:="">
<span style="font-family: Arial, Helvetica, sans-serif; font-size: x-small;">Once you are able to log in to the console, you will likely want to configure the remote KVM. This is slightly more complex:</span></div>
<ol 0px="" 10px="" margin:="">
<li><span style="font-family: Arial, Helvetica, sans-serif; font-size: x-small;">Navigate to the vKVM settings and click the Launch link, then click "Launch Java KVM Client", this should launch a JNLP file with javaws. However, since Java 1.7.0_51, self-signed code can not be executed. The work around create a file: ~/.java/deployment/security/exception.sites and add the following lines:</span><ol style="list-style-type: lower-alpha; margin: 0px;">
<li><span style="font-family: Arial, Helvetica, sans-serif; font-size: x-small;">http://<bmc_ip host=""></bmc_ip></span></li>
<li><span style="font-family: Arial, Helvetica, sans-serif; font-size: x-small;">https://<bmc_ip host=""></bmc_ip></span></li>
</ol>
</li>
<li><span style="font-family: Arial, Helvetica, sans-serif; font-size: x-small;">Now, when you run the JNLP KVM client, you will be allowed to authorize the execution of self signed code.</span><span style="font-family: Arial, Helvetica, sans-serif; font-size: x-small; text-align: center;"> </span></li>
</ol>
<div 0px="" 10px="" argin-top:="" padding:="">
<span style="font-family: Arial, Helvetica, sans-serif; font-size: x-small;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif; font-size: x-small;"><br /></span></div>
<div 0px="" 10px="" margin-top:="" padding:="">
<span style="font-family: Arial, Helvetica, sans-serif; font-size: x-small;">If you have configured everything correctly you should be able to see the BIOS where we left off in Step 4 above:</span><br />
<span style="font-family: Arial, Helvetica, sans-serif; font-size: x-small;"><br /></span></div>
<div 0px="" 10px="" margin-top:="" padding:="">
<span class="confluence-embedded-file-wrapper" style="display: inline-block; font-family: Arial, Helvetica, sans-serif; font-size: x-small; margin-left: 2px; margin-right: 2px; max-width: 100%; position: relative;"><img class="confluence-embedded-image" data-base-url="https://ccristudio.jira.com/wiki" data-image-src="/wiki/download/attachments/64749639/image2014-7-11%2013%3A29%3A49.png?version=1&modificationDate=1405099790076&api=v2" data-linked-resource-container-id="64749639" data-linked-resource-container-version="4" data-linked-resource-default-alias="image2014-7-11 13:29:49.png" data-linked-resource-id="64946185" data-linked-resource-type="attachment" data-linked-resource-version="1" height="546" src="https://ccristudio.jira.com/wiki/download/attachments/64749639/image2014-7-11%2013%3A29%3A49.png?version=1&modificationDate=1405099790076&api=v2" style="border: 0px; cursor: pointer; display: block; margin: 0px 2px; max-width: calc(100% - 4px); padding: 0px;" width="640" /></span></div>
<h2 0px="" 30px="" id="HowtoconfigureDellBMC-vMedia" margin:="" padding:="">
<span style="font-family: Arial, Helvetica, sans-serif; font-size: x-small;">
vMedia</span></h2>
<div 0px="" 10px="" margin-top:="" padding:="">
<span style="font-family: Arial, Helvetica, sans-serif; font-size: x-small;">vMedia is used to map the server's CD ROM drive to your PC's CDROM, or a disk image (*.iso,*.dmg). It is executed from the web console as well.</span></div>
<div style="background-color: white; color: #333333; line-height: 20px; margin-top: 10px; padding: 0px;">
<span class="confluence-embedded-file-wrapper" style="display: inline-block; font-family: Arial, Helvetica, sans-serif; font-size: x-small; margin-left: 2px; margin-right: 2px; max-width: 100%; position: relative;"><img class="confluence-embedded-image" data-base-url="https://ccristudio.jira.com/wiki" data-image-src="/wiki/download/attachments/64749639/image2014-7-12%2023%3A29%3A55.png?version=1&modificationDate=1405222191716&api=v2" data-linked-resource-container-id="64749639" data-linked-resource-container-version="4" data-linked-resource-default-alias="image2014-7-12 23:29:55.png" data-linked-resource-id="64946186" data-linked-resource-type="attachment" data-linked-resource-version="1" src="https://ccristudio.jira.com/wiki/download/attachments/64749639/image2014-7-12%2023%3A29%3A55.png?version=1&modificationDate=1405222191716&api=v2" style="border: 0px; cursor: pointer; display: block; margin: 0px 2px; max-width: calc(100% - 4px); padding: 0px;" /></span></div>
<div>
<span style="font-family: Arial, Helvetica, sans-serif; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: Arial, Helvetica, sans-serif; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: Arial, Helvetica, sans-serif; font-size: x-small;">Additional Information:</span></div>
<div>
<span style="font-family: Arial, Helvetica, sans-serif; font-size: x-small;"><br /></span></div>
<span style="font-family: Arial, Helvetica, sans-serif; font-size: x-small;">
Use the PowerEdge BMC getting started guide: <a data-linked-resource-container-id="64749639" data-linked-resource-container-version="4" data-linked-resource-default-alias="poweredge-c6105_User's Guide_en-us.pdf" data-linked-resource-id="64946183" data-linked-resource-type="attachment" data-linked-resource-version="1" data-nice-type="PDF Document" href="ftp://ftp.dell.com/Manuals/all-products/esuprt_ser_stor_net/esuprt_cloud_products/poweredge-c6105_User%27s%20Guide_en-us.pdf" style="color: #326ca6; text-decoration: none;">poweredge-c6105_User's Guide_en-us.pdf</a></span><br />
<div style="margin-top: 10px; padding: 0px;">
<span style="font-family: Arial, Helvetica, sans-serif; font-size: x-small;">If you get an error running the KVM 'login denied/not authroized', edit the username/password inside the JNLP file. Change it to 'root/root' or whatever the security credentials are configured as. Then relaunch Java KVM ($ javaws ~/Downloads/viewer.jnlp)</span></div>
<div style="margin-top: 10px; padding: 0px;">
<span style="font-family: Arial, Helvetica, sans-serif; font-size: x-small;"><a class="external-link" href="http://docs.oracle.com/javase/7/docs/technotes/guides/jweb/jcp/properties.html" rel="nofollow" style="color: #326ca6; text-decoration: none;">http://docs.oracle.com/javase/7/docs/technotes/guides/jweb/jcp/properties.html</a> - Info about self signed code execution in jre 1.7.0 update 51</span></div>
<div style="margin-top: 10px; padding: 0px;">
<span style="font-family: Arial, Helvetica, sans-serif; font-size: x-small;"> The Advocent KVM and vMedia require port 2068 to be accessible from the client to the server. (This means that if there is a firewall between the client and the server the client intends to connect to, an exception must be made from the client to the destination port 2068 on the server.</span></div>
Champion Of Cyrodiilhttp://www.blogger.com/profile/06792737627887393792noreply@blogger.com1tag:blogger.com,1999:blog-6167470039524068257.post-80615218951078649902015-01-26T09:10:00.000-08:002015-03-04T07:32:31.339-08:00Upgrading Openstack with FuelDISCLAIMER: This is a development environment as well as a work in progress. Do not attempt this on a 'production' system without going through the process first on a non-production system and learning how to trouble shoot the system at various points.<br />
<br />
Also, this is an 'in place' upgrade. Which means you need to have approximately less than 49% of your hardware resources used, so you can create a new environment and migrate everything. Unless you only plan to migrate a few things. However, this is an entirely different subject.<br />
<br />
I have tried to break this process down in to the following steps:<br />
<br />
<ol>
<li>Plan Upgrade</li>
<li>Migrate Instances(VMs) to free up additional nodes.</li>
<li>Remove free Node's Ceph OSD from Ceph Cluster</li>
<li>Clean up Neutron agents.</li>
<li>Disable Nova Services</li>
<li>Upgrade your Fuel Server</li>
<li>Deploy new Environment (Juno here)</li>
<li>Export Volumes from Old (Icehouse) Environment</li>
<li>Import Volumes as Images and boot new VMs w/ new Volume</li>
<li>Repeat Step 9 for all instances to keep, and Delete the old Environment</li>
</ol>
<br />
<br />
We have been running a "steady" instance of Fuel 5.1 with 3 controllers configured with "HA" (High-Availability) and 4 Nodes operating as both CEPH and Compute nodes.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://4.bp.blogspot.com/-MUpPOb6qHeI/VMZu5PuxJ9I/AAAAAAAAAjg/H8LhSFnt3Ik/s1600/Original%2BIcehouse%2BCluster.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://4.bp.blogspot.com/-MUpPOb6qHeI/VMZu5PuxJ9I/AAAAAAAAAjg/H8LhSFnt3Ik/s1600/Original%2BIcehouse%2BCluster.png" height="423" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
We started finding small timeout issues and bugs w/ 5.1, so decided it was time to upgrade. 6.0 is out, so we had to download the 5.1.1 update, and the 6.0 update. And run them both in order. Here are the links to the software, and installation guide(s). Please be advised, this information is current as of January 26, 2015.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<a href="https://www.fuel-infra.org/">https://www.fuel-infra.org/</a> - Latest Download. (Fuel 6.0 for me today.)</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="font-size: large;"><br /></span></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="font-size: large;">Step 1.) Plan</span></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
Our plan is to decommission a single Compute, Storage - Ceph OSD node, and a single Controller. Fuel will not like running HA with 2 controllers. But it should be okay to deal with it while we migrate our environment.</div>
<div class="separator" style="clear: both; text-align: left;">
<span style="font-size: large;"><br /></span></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="font-size: large;">Step 2.) Migrate instances and free up a node.</span></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<a href="http://docs.openstack.org/admin-guide-cloud/content/section_live-migration-usage.html" target="_blank">Live Migration Guide</a> - Using this guide, I was able to determine the instances I had running on Node18, and migrate them to a node that had less.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both;">
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">[root@node-10 ~]# nova host-describe node-18.ccri.com</span></div>
<div class="separator" style="clear: both;">
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">+------------------+----------------------------------+-----+-----------+---------+</span></div>
<div class="separator" style="clear: both;">
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">| HOST | PROJECT | cpu | memory_mb | disk_gb |</span></div>
<div class="separator" style="clear: both;">
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">+------------------+----------------------------------+-----+-----------+---------+</span></div>
<div class="separator" style="clear: both;">
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">| node-18.ccri.com | (total) | 24 | 72492 | 8731 |</span></div>
<div class="separator" style="clear: both;">
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">| node-18.ccri.com | (used_now) | 13 | 27136 | 260 |</span></div>
<div class="separator" style="clear: both;">
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">| node-18.ccri.com | (used_max) | 10 | 20480 | 200 |</span></div>
<div class="separator" style="clear: both;">
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">| node-18.ccri.com | dc25784fc9d94e58b3887045756cf9e8 | 8 | 16384 | 160 |</span></div>
<div class="separator" style="clear: both;">
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">| node-18.ccri.com | 0dc5c66d16b04d48b07c868cc195f46a | 2 | 4096 | 40 |</span></div>
<div class="separator" style="clear: both;">
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">+------------------+----------------------------------+-----+-----------+---------+</span></div>
<div class="separator" style="clear: both;">
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">[root@node-10 ~]# nova list --host node-18.ccri.com --all-tenants</span></div>
<div class="separator" style="clear: both;">
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">+--------------------------------------+----------+--------+------------+-------------+-------------------------------------+</span></div>
<div class="separator" style="clear: both;">
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">| ID | Name | Status | Task State | Power State | Networks |</span></div>
<div class="separator" style="clear: both;">
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">+--------------------------------------+----------+--------+------------+-------------+-------------------------------------+</span></div>
<div class="separator" style="clear: both;">
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"><b>| b58af781-bb57-4c35-bbe5-4153e2d4bb6e | alfresco | ACTIVE | - | Running | net04=192.168.111.10, 192.168.3.128 |</b></span></div>
<div class="separator" style="clear: both;">
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">| 2006d7db-d18e-4390-ae1c-40dd77644853 | hannibal | ACTIVE | - | Running | ONR=172.16.0.39, 192.168.3.161 |</span></div>
<div class="separator" style="clear: both;">
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"></span></div>
<div class="separator" style="clear: both;">
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">+--------------------------------------+----------+--------+------------+-------------+-------------------------------------+</span></div>
<div>
<br /></div>
<div>
In my case, node-15 had the least number of instance, so I decided to migrate the two instance on 18 over to 15; starting with 'alfresco'.</div>
<div>
<br /></div>
<div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">[root@node-10 ~]# nova live-migration b58af781-bb57-4c35-bbe5-4153e2d4bb6e node-15.ccri.com</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"><b>ERROR: HTTPConnectionPool(host='192.168.3.100', port=8774): Max retries exceeded with url: /v2/0dc5c66d16b04d48b07c868cc195f46a/servers/b58af781-bb57-4c35-bbe5-4153e2d4bb6e/action (Caused by <class httplib.badstatusline="">: )</class></b></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"><br /></span></div>
<div>
<br />
Notice the ERROR message. I believe this is because the migration took longer than expected. However, I verified through the nova list command, as well as the Horizon UI that the server was still migrating hosts, so i waited. It finished within 5 minutes, so i then verified:</div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">[root@node-10 ~]# nova list --host node-18.ccri.com --all-tenants</span></div>
</div>
<div>
<div style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">
+--------------------------------------+----------+--------+------------+-------------+--------------------------------+</div>
<div style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">
| ID | Name | Status | Task State | Power State | Networks |</div>
<div style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">
+--------------------------------------+----------+--------+------------+-------------+--------------------------------+</div>
<div style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">
| 2006d7db-d18e-4390-ae1c-40dd77644853 | hannibal | ACTIVE | - | Running | ONR=172.16.0.39, 192.168.3.161 |</div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">+--------------------------------------+----------+--------+------------+-------------+--------------------------------+</span></div>
</div>
<div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">[root@node-10 ~]# nova list --host node-15.ccri.com --all-tenants</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">+--------------------------------------+-----------------+--------+------------+-------------+--------------------------------------+</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">| ID | Name | Status | Task State | Power State | Networks |</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">+--------------------------------------+-----------------+--------+------------+-------------+--------------------------------------+</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">| 2e1057ee-48d5-4b7f-aa9e-14b0103535ec | Mantis | ACTIVE | - | Running | ONR=172.16.0.10, 192.168.3.138 |</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"><b>| b58af781-bb57-4c35-bbe5-4153e2d4bb6e | alfresco | ACTIVE | - | Running | net04=192.168.111.10, 192.168.3.128 |</b></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">| 655fba2a-9867-4305-935c-e6b3c3a84368 | docker-registry | ACTIVE | - | Running | net04=192.168.111.7, 192.168.3.130 |</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">| f368bab9-e054-4bda-84ee-e5633e6381cb | docker01 | ACTIVE | - | Running | DS Network=172.16.0.4, 192.168.3.140 |</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">| b336499d-7314-464f-9f98-ee1ed0ddd787 | inventory | ACTIVE | - | Running | net04=192.168.111.8, 192.168.3.131 |</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">| 1b57d04c-29c7-4a1b-8cac-114f491ec5d3 | onr-node-4 | ACTIVE | - | Running | ONR=172.16.0.54, 192.168.3.167 |</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">+--------------------------------------+-----------------+--------+------------+-------------+--------------------------------------+</span></div>
</div>
<div>
<br /></div>
<div>
<br /></div>
<div>
Also notice, the public IP did not change. This is good. :) Repeat this process to free up a second node to support ceph install on juno, or find a new server in your budget to use.<br />
<br />
<span style="font-size: large;">Step 3.) Remove the Ceph OSD from Ceph cluster</span><br />
<br />
Since I am removing Node-18, I will remove the ceph instance from OSD via Node-18!<br />
<br />
<a href="http://ceph.com/docs/master/rados/operations/add-or-rm-osds/#take-the-osd-out-of-the-cluster" target="_blank">http://ceph.com/docs/master/rados/operations/add-or-rm-osds/#take-the-osd-out-of-the-cluster</a><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"><br /></span>
List some pools for sanity.<br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"><br /></span>
<span style="font-family: 'Courier New', Courier, monospace; font-size: xx-small;"> [root@node-18 ~]# ceph osd lspools</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">0 data,1 metadata,2 rbd,3 images,4 volumes,5 .rgw.root,6 compute,7 .rgw.control,8 .rgw,9 .rgw.gc,10 .users.uid,11 .rgw.buckets.index,12 .rgw.buckets,</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"><br /></span>
Determine which OSD ID this node has. #6 in this case<br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"><br /></span>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">[root@node-18 ~]# ps -ef | grep ceph</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">root 3258 1 2 2014 ? 1-01:46:17 /usr/bin/ceph-osd -i 6 --pid-file /var/run/ceph/osd.6.pid -c /etc/ceph/ceph.conf --cluster ceph</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">root 11490 10726 0 17:25 pts/0 00:00:00 grep ceph</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"><br /></span>
Mark it for removal<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"><br /></span>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"><br /></span>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">[root@node-18 ~]# ceph osd out 6</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">marked out osd.6. </span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"><br /></span>
Watch the rebalance happen<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"><br /></span>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"><br /></span>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">[root@node-18 ~]# ceph -w</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> cluster 994f6ed1-69c0-4e8b-8c76-fc1186c7eda5</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> health HEALTH_WARN mon.node-10 low disk space; mon.node-12 low disk space; mon.node-13 low disk space</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> monmap e3: 3 mons at {node-10=10.10.20.3:6789/0,node-12=10.10.20.5:6789/0,node-13=10.10.20.6:6789/0}, election epoch 676, quorum 0,1,2 node-10,node-12,node-13</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> osdmap e217: 7 osds: 7 up, 6 in</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> pgmap v6442467: 5312 pgs, 13 pools, 1301 GB data, 301 kobjects</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> 2304 GB used, 5895 GB / 8199 GB avail</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> 1 active+clean+scrubbing+deep</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> 5311 active+clean</span><br />
<br /></div>
<div>
You will see lot of entries that describe what is happening. Most importantly something like this:<br />
<br />
9854/630322 objects degraded (1.563%)<br />
<br />
This means that ceph's RBD objects (which keep your openstack data), do not have enough replicas for 9854 objects. It will copy the replica to another host, so you will have to wait until all objects that were on your OSD node are rebalanced. This will utilize a lot of network I/O, and your active VMs will suffer. So warn your users before doing this.<br />
<br />
Again, ensure this is done on any additonal nodes you want to delete.</div>
<div>
<br />
<span style="font-size: large;">Step 4.) Cleanup Neutron</span><br />
<div>
<br /></div>
<div>
You may notice that once your node is gone, there are some stale neutron agents marked dead.</div>
<div>
<br /></div>
<div>
The following command will list all dead (xxx) agents and show details. Change 'agent-show' to 'agent-delete' to remove them permenantly:</div>
<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">for i in $(neutron agent-list | grep "xxx" | awk '{print $2}'); do neutron agent-show $i; done;</span><br />
<br />
<span style="font-size: large;">Step 5.) Cleanup Nova Services</span><br />
<br />
Just like the neutron services, the nova services on your old node may still show up in horizon as 'down'. You can use something like the command below to disable them.<br />
<br />
for i in $(nova service-list | grep node-13 | awk '{print $2}'); do nova service-disable node-13.ccri.com $i; done;<br />
<br />
<div>
</div>
I have did not figure out how to delete the services. But it doesn't really matter, because I will be deleting the entire environment once the upgrade and migrations are complete.<br />
<div style="-webkit-text-stroke-width: 0px; color: black; font-family: 'Times New Roman'; font-size: medium; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px;">
<div style="margin: 0px;">
<br /></div>
</div>
<br />
<span style="font-size: large;">Step 6.) Upgrade to Fuel 6.0 if you didn't already.</span><br />
<br /></div>
<div>
Once your instances have been all migrated. You should be able to use the Fuel UI and decomission the node and one controller. Run the update.sh either before or after. I did it before migrating instances.<br />
<br />
<span style="font-size: large;">Step 7.) Create the new Environment with Fuel UI.</span></div>
<div>
<br /></div>
<div>
You should now have a free controller and nodes to build a Juno Openstack environment to start migrating your instances from the old Icehouse Openstack environment. Hopefully with virtually 0 downtime.</div>
<div>
<br /></div>
<div>
Ceph installation requires at least 2 nodes.<br />
<br />
<div>
<span style="font-size: large;">Step 8.) Export Volumes from Old Environment</span><br />
<br />
There are likely a variety of ways to import/export volumes in openstack. I have found the following method works well.<br />
<br />
First, find a place on your old controller w/ extra disk. Generally /var/lib/mongo has a lot of space w/ default partitioning. Locate the UUID for a volume using nova or cinder list. instance IDs are used for the ephemeral disks, volume IDs are used for 'volume' disks. Make sure the Instance using the disk is shut off.<br />
<br />
Export it with rbd, then compress it to qcow2 so you can pass it over the Fuel network to your other environment's controller. In this example, I am exporting a 'Volume' as 'raw' disk, then converting it.<br />
<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">[root@node-10 mongo]# rbd export --pool=volumes volume-0f2a87ec-74c5-4356-a4e7-12fffd6fe5ea docker-registry.raw </span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">Exporting image: 100% complete...done.</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">[root@node-10 mongo]# qemu-img convert -f raw -O qcow2 ./docker-registry.raw docker-registry.qcow2</span><br />
<br />
To support SCP, on the old environment controller modify /etc/ssh/sshd_config and set PasswordAuthentication to 'yes' which is at the bottom of the file. (Also int he middle of the file, but commented out). Then # useradd temp and set the password, # passwd temp<br />
#service sshd restart, and you should now be able to scp the data from the new controller.<br />
<br />
<span style="font-size: large;">Step 9.) Import Volumes as Images and launch</span><br />
<br />
Here, my new Juno environment's controller is 'node-35'<br />
<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">[root@node-35 ~]# scp temp@node-10:/var/lib/mongo/docker-registry.qcow2 . </span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">Warning: Permanently added 'node-10,10.20.0.3' (RSA) to the list of known hosts.</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">temp@node-10's password:</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">docker-registry.qcow2 65% 7489MB 90.3MB/s 00:21 ETA</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span>
<span style="font-family: inherit;">Get the byte size first:</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">[root@node-35 ~]# ls -al docker-registry.qcow2 </span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">-rw-r--r-- 1 root root 11334057984 Feb 11 16:34 docker-registry.qcow2</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span>
And now, import it as an image with glance:<br />
<br />
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">[root@node-35 ~]#</span><span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> </span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;">glance image-create --size </span><span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">11334057984</span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> --name docker-registry --store rbd --disk-format qcow2 --container-format bare --file ./docker-registry.qcow2</span><br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://1.bp.blogspot.com/-o5lJ8l3lg-4/VNuHPj_h3GI/AAAAAAAAAkc/f9fLeCOX2zM/s1600/glance-import.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" src="http://1.bp.blogspot.com/-o5lJ8l3lg-4/VNuHPj_h3GI/AAAAAAAAAkc/f9fLeCOX2zM/s1600/glance-import.png" height="185" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Importing into Glance while watching Ceph/rbd</td></tr>
</tbody></table>
<div style="text-align: center;">
<br /></div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">+------------------+--------------------------------------+</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">| Property | Value |</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">+------------------+--------------------------------------+</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">| checksum | a209fafa8ae5369e0a93b30e41c4e27c |</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">| container_format | bare |</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">| created_at | 2015-02-11T16:40:09 |</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">| deleted | False |</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">| deleted_at | None |</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">| disk_format | qcow2 |</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">| id | 22209ea8-2287-425b-9e45-c79ec210d380 |</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">| is_public | False |</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">| min_disk | 0 |</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">| min_ram | 0 |</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">| name | docker-registry |</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">| owner | aeea9a5fd7284450a3468915980a8c45 |</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">| protected | False |</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">| size | 11334057984 |</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">| status | active |</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">| updated_at | 2015-02-11T16:47:53 |</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">| virtual_size | None |</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">+------------------+--------------------------------------+</span><br />
<div>
<br /></div>
<br />
<span style="font-family: inherit;">At this point, you should be able to launch a new instance from a converted Image->Volume, and specify this glance image as the source, and create a new Volume specifying the size of the original volume. In my case with this docker registry, it was 100GB, even though qcow compressed it down to 11GB.</span><br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://3.bp.blogspot.com/-9qkfOO-UEWQ/VNuK47hR1uI/AAAAAAAAAko/WGyXo-S_zHI/s1600/image-to-volume.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" src="http://3.bp.blogspot.com/-9qkfOO-UEWQ/VNuK47hR1uI/AAAAAAAAAko/WGyXo-S_zHI/s1600/image-to-volume.png" height="129" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Create Volume from Image</td></tr>
</tbody></table>
<div style="text-align: center;">
<span style="font-family: inherit;"><br /></span></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://4.bp.blogspot.com/-HfRj_3ePnN8/VNuLAuy4PFI/AAAAAAAAAkw/Y8j8pemZhCo/s1600/image-to-volume2.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" src="http://4.bp.blogspot.com/-HfRj_3ePnN8/VNuLAuy4PFI/AAAAAAAAAkw/Y8j8pemZhCo/s1600/image-to-volume2.png" height="400" width="387" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Create Volume from Image with Specified Size of Original Volume (Not qcow size!)</td></tr>
</tbody></table>
<div style="text-align: center;">
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://1.bp.blogspot.com/-5KiEsxmethA/VNuTImu5i8I/AAAAAAAAAlA/yVHaXKUA_Tc/s1600/boot-from-volume.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" src="http://1.bp.blogspot.com/-5KiEsxmethA/VNuTImu5i8I/AAAAAAAAAlA/yVHaXKUA_Tc/s1600/boot-from-volume.png" height="396" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Booting instance from new Volume</td></tr>
</tbody></table>
<span style="font-family: inherit;"><br /></span>
<br />
<div style="text-align: left;">
<span style="font-family: inherit;">If at any point the UI has an error. Just watch the osd pool stats # watch ceph osd pool stats volumes</span></div>
<div style="text-align: left;">
<span style="font-family: inherit;">You should see client io that is pretty heavy. When it is done, you can refresh your volumes on horizon UI and it should be there as 'Available'.</span></div>
</div>
<span style="font-family: inherit;"><br /></span>
<span style="font-size: large;">Step 10.) "Rinse and Repeat"</span><br />
<span style="font-family: inherit;"><br /></span>
Go ahead and repeat step 8/9 for all the instances you want migrated. Set up your public IPs, update your external DNS entries, etc. And wait a day to make sure things are stable. Afterwards go ahead and delete the old environment, add the free nodes to your new environment, and migrate some instances to lighten the load on your first couple of nodes, and you should be good to go!<br />
<br />
<br /></div>
</div>
Champion Of Cyrodiilhttp://www.blogger.com/profile/06792737627887393792noreply@blogger.com0tag:blogger.com,1999:blog-6167470039524068257.post-32388093660588209172015-01-19T12:45:00.002-08:002015-01-19T12:46:56.301-08:00Docker Python APIHere is an example Python script using the docker-py API. In this example, I start 3 containers. One with Accumulo, one with Apache YARN, and one with Geoserver. I am also linking the containers so that they have hosts file entries to support the hostname lookups.<br />
<br />
additionally i have declared some volumes that i bind to the host's home folder under /geomesa-docker-volumes/*<br />
<br />
If you get version mismatch errors, just modify the version in the get_client_unsecure function.<br />
<br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">#!/usr/bin/env python</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"># The unsecure client requires that your Docker daemon is listening on port 5555 in addition to the default unix socket.</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"># DOCKER_OPTS="-H unix:///var/run/docker.sock -H tcp://127.0.0.1:5555"</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"># $ sudo service docker(.io) restart</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"><br /></span>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">__author__ = 'championofcyrodiil'</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"><br /></span>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">import docker</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">import getpass</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">from subprocess import call</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"><br /></span>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">geoserver_image = "user:geoserver"</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">accumulo_image = "user:accumulo"</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">yarn_image = "user:yarn"</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">remote_docker_daemon_host = "127.0.0.1"</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">unsecure_docker_port = 5555</span><br />
<br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">def get_client_unsecure(host, port):</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> client = docker.Client(base_url="http://%s:%s" % (host, port), version="1.10")</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> return client</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"><br /></span>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"><br /></span>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">if __name__ == '__main__':</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> # unsecured connection on localhost (127.0.0.1)</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> dc = get_client_unsecure(remote_docker_daemon_host, unsecure_docker_port)</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"><br /></span>
<span style="font-family: 'Courier New', Courier, monospace; font-size: xx-small;">def start_geomesa(user):</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> #accumulo container</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> accumulo_volumes = ['/opt/accumulo/accumulo-1.5.2/lib/ext/', '/data-dir/', '/data']</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> accumulo_container = \</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> dc.create_container(image=accumulo_image,</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> name=(user + 's-accumulo'),</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> tty=True,</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> stdin_open=True,</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> hostname='accumulo',</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> ports=[2181, 22, 50070, 50095, 50075, 9000, 9898, 3614],</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> volumes=accumulo_volumes,</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> mem_limit="4g")</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"><br /></span>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> accumulo_binds = {</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> '/home/' + getpass.getuser() + '/geomesa-docker-volumes/accumulo-libs':</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> {</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> 'bind': '/opt/accumulo/accumulo-1.5.2/lib/ext/',</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> 'ro': False</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> },</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> '/home/' + getpass.getuser() + '/geomesa-docker-volumes/accumulo-data':</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> {</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> 'bind': '/data-dir/',</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> 'ro': False</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> },</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> '/home/' + getpass.getuser() + '/geomesa-docker-volumes/hdfs-data':</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> {</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> 'bind': '/data/',</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> 'ro': False</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> }</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> }</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> dc.start(accumulo_container, publish_all_ports=True, binds=accumulo_binds)</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"><br /></span>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> #YARN CONTAINER</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> yarn_container = dc.create_container(image=yarn_image,</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> name=(user + 's-yarn'),</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> stdin_open=True,</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> tty=True,</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> hostname='yarn',</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> ports=[8088, 8042, 22], mem_limit="2g")</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"><br /></span>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> link = {(user + 's-accumulo'): 'accumulo'}</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> dc.start(yarn_container,</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> publish_all_ports=True,</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> links=link)</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"><br /></span>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> #geoserver container</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> geoserver_container = dc.create_container(image=geoserver_image,</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> name=(user + 's-geoserver'),</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> stdin_open=True,</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> tty=True,</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> hostname='geoserver',</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> ports=[8080, 22, 7979], mem_limit="2g")</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> link = {(user + 's-accumulo'): 'accumulo', (user + 's-yarn'): 'yarn'}</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> dc.start(geoserver_container,</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> publish_all_ports=True,</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> links=link)</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"><br /></span>
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">start_geomesa('test')</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">call("./geomesa_info.py")</span><br />
<div>
<br /></div>
Champion Of Cyrodiilhttp://www.blogger.com/profile/06792737627887393792noreply@blogger.com0