marfarma - Developer IT

MySql Data Loss - post mortem analysis - RackSpace Cloud Server

- by marfarma

After a recent 'emergency migration' of a RS cloud server, the mysql databases on our server snapshot image proved to be days out of date from the backup date. And yet files that were uploaded through the impacted webapp had been written to the file system. Related metadata that was written to the database was lost, but the files themselves were backed-up. Once I was able to manually access the mysql data files before the mysql server started (server was configured to start mysql on boot), I was able to see that the update time for ib_logfile1, ib_logfile0 and ibdata1 was days old. As with this poster, mysql data loss after server crash, it's as if some caching controller had told the OS / mysql server that it had committed data that was still in cache, and it was lost instead of flushed. I can't quite wrap my head around how the uploaded files got written but the database data did not. I would have thought that any cache would have flushed system wide, rather than process by process. Any suggestions as to how this might have happened?

Read the article

Virtual Lan on the Cloud -- Help Confirm my understanding?

- by marfarma

[Note: Tried to post this over at ServerFault, but I don't have enough 'points' for more than one link. Powers that be, move this question over there.] Please give this a quick read and let me know if I'm missing something before I start trying to make this work. I'm not a systems admin professional, and I'd hate to end up banging my head into the wall if I can avoid it. Goals: Create a 'road-warrior' capable star shaped virtual LAN for consultants who spend the majority of their time on client sites, and who's firm has no physical network or servers. Enable CIFS access to a cloud-server based installation of Alfresco Allow Eventual implementation of some form of single-sign-on ( OpenLDAP server ) access to Alfresco and other server applications implemented in the future Given: All Servers will live in the public internet cloud (Rackspace Cloud Servers) OpenVPN Server will be a Linux disto, probably Ubuntu 9.x, installed on same server as Alfresco (at least to start) Staff will access server applications and resources from client sites, hotels, trains, planes, coffee shops or their homes over various ISP, using their company laptops or personal home desktops. Based on my Research thus far, to accomplish this, I'll need: OpenVPN with Bridging Enabled to create a star shaped "virtual" LAN http://openvpn.net/index.php/open-source/documentation/miscellaneous/76-ethernet-bridging.html A Road Warrior Network Configuration, as described in this Shorewall article (lower down the page) http://www.shorewall.net/OPENVPN.html Configure bridge addressesing (probably DHCP) http://openvpn.net/index.php/open-source/faq.html#bridge-addressing Configure CIFS / Samba to accept VPN IP address http://serverfault.com/questions/137933/howto-access-samba-share-over-vpn-tunnel Set up Client software, with keys configured for access (potentially through a OpenVPN-Sa client portal) http://www.openvpn.net/index.php/access-server/download-openvpn-as/221-installation-overview.html

Read the article

iptables & allowed port refusing connection

- by marfarma

Can you see what I'm doing wrong? On Ubuntu Server 9.1, I'm attempting to allow traffic on port 1143 for a non-privileged IMAP host. Connection is refused when testing with telnet example.com 1143 but connection is allowed testing with telnet example.com 80 from my pc to remote internet hosted server. Both rules appear identical and are located near each other with no rules rejecting connections intervening in the rules file. I can't figure it out. iptables -L returns this: Chain INPUT (policy ACCEPT) target prot opt source destination ACCEPT all -- anywhere anywhere REJECT all -- anywhere 127.0.0.0/8 reject-with icmp-port-unreachable ACCEPT all -- anywhere anywhere state RELATED,ESTABLISHED ACCEPT tcp -- anywhere anywhere tcp dpt:www ACCEPT tcp -- anywhere anywhere tcp dpt:https ACCEPT tcp -- anywhere anywhere tcp dpt:http-alt ACCEPT tcp -- anywhere anywhere tcp dpt:7070 ACCEPT tcp -- anywhere anywhere tcp dpt:1143 ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:ssh ACCEPT icmp -- anywhere anywhere icmp echo-request LOG all -- anywhere anywhere limit: avg 5/min burst 5 LOG level debug prefix `iptables denied: ' REJECT all -- anywhere anywhere reject-with icmp-port-unreachable Chain FORWARD (policy ACCEPT) target prot opt source destination REJECT all -- anywhere anywhere reject-with icmp-port-unreachable Chain OUTPUT (policy ACCEPT) target prot opt source destination ACCEPT all -- anywhere anywhere and my rules file contains this: # Generated by iptables-save v1.4.4 on Wed May 26 19:08:34 2010 *nat :PREROUTING ACCEPT [3556:217296] :POSTROUTING ACCEPT [6909:414847] :OUTPUT ACCEPT [6909:414847] -A PREROUTING -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 8080 -A PREROUTING -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 8080 COMMIT # Completed on Wed May 26 19:08:34 2010 # Generated by iptables-save v1.4.4 on Wed May 26 19:08:34 2010 *filter :INPUT ACCEPT [1:52] :FORWARD ACCEPT [0:0] :OUTPUT ACCEPT [1:212] -A INPUT -i lo -j ACCEPT -A INPUT -d 127.0.0.0/8 ! -i lo -j REJECT --reject-with icmp-port-unreachable -A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT -A INPUT -p tcp -m tcp --dport 80 -j ACCEPT -A INPUT -p tcp -m tcp --dport 443 -j ACCEPT -A INPUT -p tcp -m tcp --dport 8080 -j ACCEPT -A INPUT -p tcp -m tcp --dport 7070 -j ACCEPT -A INPUT -p tcp -m tcp --dport 1143 -j ACCEPT -A INPUT -p tcp -m state --state NEW -m tcp --dport 22 -j ACCEPT -A INPUT -p icmp -m icmp --icmp-type 8 -j ACCEPT -A INPUT -m limit --limit 5/min -j LOG --log-prefix "iptables denied: " --log-level 7 -A INPUT -j REJECT --reject-with icmp-port-unreachable -A FORWARD -j REJECT --reject-with icmp-port-unreachable -A OUTPUT -j ACCEPT COMMIT # Completed on Wed May 26 19:08:34 2010

Read the article

Virtual Lan on the Cloud -- Help Confirm my understanding?

- by marfarma

[Note: Tried to post this over at ServerFault, but I don't have enough 'points' for more than one link. Powers that be, move this question over there.] Please give this a quick read and let me know if I'm missing something before I start trying to make this work. I'm not a systems admin professional, and I'd hate to end up banging my head into the wall if I can avoid it. Goals: Create a 'road-warrior' capable star shaped virtual LAN for consultants who spend the majority of their time on client sites, and who's firm has no physical network or servers. Enable CIFS access to a cloud-server based installation of Alfresco Allow Eventual implementation of some form of single-sign-on ( OpenLDAP server ) access to Alfresco and other server applications implemented in the future Given: All Servers will live in the public internet cloud (Rackspace Cloud Servers) OpenVPN Server will be a Linux disto, probably Ubuntu 9.x, installed on same server as Alfresco (at least to start) Staff will access server applications and resources from client sites, hotels, trains, planes, coffee shops or their homes over various ISP, using their company laptops or personal home desktops. Based on my Research thus far, to accomplish this, I'll need: OpenVPN with Bridging Enabled to create a star shaped "virtual" LAN http://openvpn.net/index.php/open-source/documentation/miscellaneous/76-ethernet-bridging.html A Road Warrior Network Configuration, as described in this Shorewall article (lower down the page) http://www.shorewall.net/OPENVPN.html Configure bridge addressesing (probably DHCP) http://openvpn.net/index.php/open-source/faq.html#bridge-addressing Configure CIFS / Samba to accept VPN IP address http://serverfault.com/questions/137933/howto-access-samba-share-over-vpn-tunnel Set up Client software, with keys configured for access (potentially through a OpenVPN-Sa client portal) http://www.openvpn.net/index.php/access-server/download-openvpn-as/221-installation-overview.html

Read the article

Can't build gem -- native extension build fails -- can you see why?

- by marfarma

I can't figure out what is going wrong here -- any ideas?? I'm running on a Ubuntu 8.04 LTS, and have installed libxml2 and libxslt from these instructions: http://www.techsww.com/tutorials/libraries/libxml/installation/installing_libxml_on_ubuntu_linux.php http://www.techsww.com/tutorials/libraries/libxslt/installation/installing_libxslt_on_ubuntu_linux.php However, I installed the latest versions: libxslt-1.1.24 libxml2-2.7.3 The install was uneventful -------------------- I set LD_LIBRARY_PATH ---------------------------------- echo $LD_LIBRARY_PATH /usr/local/libxslt/lib: ------------- seems like the function is present -- at least based on the output of strings ------------ /usr/local/libxslt/lib$ strings * | grep ParseStylesheetDoc xsltParseStylesheetDoc xsltParseStylesheetDoc xsltParseStylesheetDoc xsltParseStylesheetDoc xsltParseStylesheetDoc xsltParseStylesheetDoc xsltParseStylesheetDoc ----------------------- But the compile still fails ---------------------------------------- sudo gem install webrat Building native extensions. This could take a while... ERROR: Error installing webrat: ERROR: Failed to build gem native extension. /usr/local/bin/ruby extconf.rb install webrat checking for iconv.h in /opt/local/include/,/opt/local/include/libxml2,/opt/local/include,/opt/local/include,/opt/local/include/libxml2,/usr/local/include,/usr/local/include/libxml2,/usr/local/include,/usr/local/include/libxml2,/usr/include,/usr/include/libxml2... yes checking for libxml/parser.h in /opt/local/include/,/opt/local/include/libxml2,/opt/local/include,/opt/local/include,/opt/local/include/libxml2,/usr/local/include,/usr/local/include/libxml2,/usr/local/include,/usr/local/include/libxml2,/usr/include,/usr/include/libxml2... yes checking for libxslt/xslt.h in /opt/local/include/,/opt/local/include/libxml2,/opt/local/include,/opt/local/include,/opt/local/include/libxml2,/usr/local/include,/usr/local/include/libxml2,/usr/local/include,/usr/local/include/libxml2,/usr/include,/usr/include/libxml2... yes checking for libexslt/exslt.h in /opt/local/include/,/opt/local/include/libxml2,/opt/local/include,/opt/local/include,/opt/local/include/libxml2,/usr/local/include,/usr/local/include/libxml2,/usr/local/include,/usr/local/include/libxml2,/usr/include,/usr/include/libxml2... yes checking for xmlParseDoc() in -lxml2... yes checking for xsltParseStylesheetDoc() in -lxslt... no libxslt is missing. try 'port install libxslt' or 'yum install libxslt-devel' *** extconf.rb failed *** Could not create Makefile due to some reason, probably lack of necessary libraries and/or headers. Check the mkmf.log file for more details. You may need configuration options. Provided configuration options: --with-opt-dir --without-opt-dir --with-opt-include --without-opt-include=${opt-dir}/include --with-opt-lib --without-opt-lib=${opt-dir}/lib --with-make-prog --without-make-prog --srcdir=. --curdir --ruby=/usr/local/bin/ruby --with-iconv-dir --without-iconv-dir --with-iconv-include --without-iconv-include=${iconv-dir}/include --with-iconv-lib --without-iconv-lib=${iconv-dir}/lib --with-xml2-dir --without-xml2-dir --with-xml2-include --without-xml2-include=${xml2-dir}/include --with-xml2-lib --without-xml2-lib=${xml2-dir}/lib --with-xslt-dir --without-xslt-dir --with-xslt-include --without-xslt-include=${xslt-dir}/include --with-xslt-lib --without-xslt-lib=${xslt-dir}/lib --with-xml2lib --without-xml2lib --with-xsltlib --without-xsltlib Gem files will remain installed in /usr/local/lib/ruby/gems/1.8/gems/nokogiri-1.3.3 for inspection. Results logged to /usr/local/lib/ruby/gems/1.8/gems/nokogiri-1.3.3/ext/nokogiri/gem_make.out

Read the article

How did this happen?? Git error? Some other fluke?

- by marfarma

Every file in this Rails project is duplicated with a -e and again with a -e-e tacked onto the end of it, like the following. It's that way in my GitHub repository too. But I can't figure out how it happened. Any clue? Google searching comes up empty. -rw-r--r--@ 1 usrname staff 959 Jan 7 02:13 Gemfile -rw-r--r-- 1 usrname staff 958 Jan 5 01:10 Gemfile-e -rw-r--r-- 1 usrname staff 958 Jan 5 01:09 Gemfile-e-e -rw-r--r-- 1 usrname staff 6650 Jan 7 02:13 Gemfile.lock -rw-r--r-- 1 usrname staff 6650 Jan 5 01:10 Gemfile.lock-e -rw-r--r-- 1 usrname staff 6650 Jan 5 01:09 Gemfile.lock-e-e lrwxr-xr-x 1 usrname staff 18 Jan 5 00:37 README.rdoc - doc/README_FOR_APP -rw-r--r-- 1 usrname staff 283 Jan 5 01:10 Rakefile -rw-r--r-- 1 usrname staff 283 Jan 5 01:10 Rakefile-e -rw-r--r-- 1 usrname staff 283 Jan 5 01:09 Rakefile-e-e drwxr-xr-x 6 usrname staff 204 Jan 5 00:37 app drwxr-xr-x 5 usrname staff 170 Jan 5 01:10 autotest drwxr-xr-x 28 usrname staff 952 Jan 5 01:15 config -rw-r--r-- 1 usrname staff 173 Jan 5 01:10 config.ru -rw-r--r-- 1 usrname staff 173 Jan 5 01:10 config.ru-e -rw-r--r-- 1 usrname staff 173 Jan 5 01:09 config.ru-e-e

Read the article

xvfb on a machine with a display, can an application run 'in the background?'

- by marfarma

I'm setting up to cron a web scraping job, using xvfb, firefox, and watir on my Mac OS X. In testing the script so far, firefox pops up visibly on the local desktop, the watir script executes, and then firefox exits (I quit firefox in my script). I'd like to set the xvfb DISPLAY such that firefox will run, but won't be seen on the local desktop, running 'in the background' so to speak. Nothing I've been able to find online discusses such a possibility - nor explains that it's not possible. Is it possible? If so, what do I need to do to make it work?

Developer IT