Search Results

Search found 8 results on 1 pages for 'journalling'.

Page 1/1 | 1 

• Run a MongoDB configuration server without 3GB of journal files

  - by Thilo

  For a production sharded MongoDB installation we need 3 configuration servers. According to the documentation "the config server mongod process is fairly lightweight and can be ran on machines performing other work". However, in the default configuration, they all have journalling enabled, and with preallocation this takes up 3 GB of disk space. I assume that the actual data and transaction volume of a config server is quite small, so that this seems a bit too much. Is there a way to (safely!) run these config servers with much less disk use for the journal? Do I need journalling at all on config servers? Can I set the journal size to be smaller?

  Read the article

• SSD: NTFS vs EXT4

  - by Joshua

  Always when I read about SSD usage under Linux, the advice is to disable journalling in Ext4 (or use Ext2), since it's too bad for your SSD. But in all articles about SSD tweaks for Windows, I never see any mentions that you should disable NTFS journalling, or that you should stick to FAT32. I know Ext4's journaling is more advanced, but is it so much more damaging to a SSD than that of NTFS? Or are Linux users just a little bit more cautious?

  Read the article

• why does chkdsk always report errors on a bad shutdown

  - by rep_movsd

  Once in a while, Windows XP hangs on my laptop (usually when going into standby or hibernate and occasionally on startup) and I have to forcefully poweroff. Ususally chkdsk never runs automatically (I thought it should know that the partitions have nit been unmounted and do that). I religiously run chkdsk without /F after bad shutdowns like this, and invariably it reports that the drive has unfixed errors and must be checked with /F and I do that, and more often than not, the chdsk that runs on startup does not report fixing anything. I have had times in the past (and not only just on this system) when not running chkdsk leads to some strange errors like files not opening even though they exist and inability to save certain files, so I make it a point to always chkdsk after bad shutdown. I never understood why this is : Isnt the whole point of a journalling filesystem like NTFS to avoid file system corruption and endless chkdsks? I even tried once disabling write caching to see if it made any difference, but to no avail....

  Read the article

• NVRAM for journals on Linux?

  - by symcbean

  I've been thinking about ways of speeding up disk I/O, and one of the bottlenecks I keep coming back to is the journal. There's an obvious benefit to using an SSD for the journal - over and above just write caching unless of course I just disable the journal with the write cache (after all devicemapper doesn't seem to support barriers). In order to get the benefits from using a BB write cache on the controller, then I'd need to disable journalling - but then the OS should try to fsck the system after an outage. Of course if the OS knows what's in the batter-backed memory then it could use it as the journal - but that means it must be exposed as a block device and only be under the control of the operating system. However I've not been able to find a suitable low-cost device (no, write-levelling for Flash is not adequate for a journal, at least one which uses Smartmedia). While there's no end of flash devices, disk/array controllers with BB write caches, so far I've not found anything which just gives me non-volatile memory addressable as a block storage device.

  Read the article

• Turn off write barriers on ext4 whiche FS is mounted

  - by user462982

  I am doing some IO intensive DB imports that run for several days now and the IO performance has dropped tremendously over times. The DB data files (log files) are on an ext4 formatted logical volume which is mounted with default options (did not specify something special in fstab). Since I just learned that ext4 enables write barriers by default: Q: Is there some way to disable write barriers online (i.e. while the file system is in use), because I cannot interrupt the import and don't want to restart it again. I am aware that write barriers might not be the only thing impeding performance it is a bad idea to have write barriers disabled on journalling file systems if data safty is important (e.g. on a production system)

  Read the article

• Usb stick too slow to benchmark?

  - by user85340

  I have a Core 2 Duo [email protected] with 3GB RAM. After some time using XUbuntu 10.10 on an 8GB stick I decided to switch to 12.04 and put it onto a 32GB stick (Transcend). I use an EXT4 with no journalling, noatime etc set. /tmp and /run is using tmpfs. And it is REALLY slow. MUCH slower than the old Xubuntu on the 8GB stick. Starting takes minutes, all applications "fade" because they respond too slow. I first thought that the NVidia graphics card is responsible for this, because there seem to be some known problems with that. Doing the adjustment (uncheck the sync checkbox) did not help. I believe the root cause is that the access to the USB stick is extremely slow. Running the read benchmark of the disk utility then brought the message "disk is too slow to benchmark"! BUT: When I do the same benchmark with the live CD I get around 20MB read performance and have a very responsive system! So how can I find out what is going one here?

  Read the article

• Is post-sudden-power-loss filesystem corruption on an SSD drive's ext3 partition "expected behavior"?

  - by Jeremy Friesner

  My company makes an embedded Debian Linux device that boots from an ext3 partition on an internal SSD drive. Because the device is an embedded "black box", it is usually shut down the rude way, by simply cutting power to the device via an external switch. This is normally okay, as ext3's journalling keeps things in order, so other than the occasional loss of part of a log file, things keep chugging along fine. However, we've recently seen a number of units where after a number of hard-power-cycles the ext3 partition starts to develop structural issues -- in particular, we run e2fsck on the ext3 partition and it finds a number of issues like those shown in the output listing at the bottom of this Question. Running e2fsck until it stops reporting errors (or reformatting the partition) clears the issues. My question is... what are the implications of seeing problems like this on an ext3/SSD system that has been subjected to lots of sudden/unexpected shutdowns? My feeling is that this might be a sign of a software or hardware problem in our system, since my understanding is that (barring a bug or hardware problem) ext3's journalling feature is supposed to prevent these sorts of filesystem-integrity errors. (Note: I understand that user-data is not journalled and so munged/missing/truncated user-files can happen; I'm specifically talking here about filesystem-metadata errors like those shown below) My co-worker, on the other hand, says that this is known/expected behavior because SSD controllers sometimes re-order write commands and that can cause the ext3 journal to get confused. In particular, he believes that even given normally functioning hardware and bug-free software, the ext3 journal only makes filesystem corruption less likely, not impossible, so we should not be surprised to see problems like this from time to time. Which of us is right? Embedded-PC-failsafe:~# ls Embedded-PC-failsafe:~# umount /mnt/unionfs Embedded-PC-failsafe:~# e2fsck /dev/sda3 e2fsck 1.41.3 (12-Oct-2008) embeddedrootwrite contains a file system with errors, check forced. Pass 1: Checking inodes, blocks, and sizes Pass 2: Checking directory structure Invalid inode number for '.' in directory inode 46948. Fix<y>? yes Directory inode 46948, block 0, offset 12: directory corrupted Salvage<y>? yes Entry 'status_2012-11-26_14h13m41.csv' in /var/log/status_logs (46956) has deleted/unused inode 47075. Clear<y>? yes Entry 'status_2012-11-26_10h42m58.csv.gz' in /var/log/status_logs (46956) has deleted/unused inode 47076. Clear<y>? yes Entry 'status_2012-11-26_11h29m41.csv.gz' in /var/log/status_logs (46956) has deleted/unused inode 47080. Clear<y>? yes Entry 'status_2012-11-26_11h42m13.csv.gz' in /var/log/status_logs (46956) has deleted/unused inode 47081. Clear<y>? yes Entry 'status_2012-11-26_12h07m17.csv.gz' in /var/log/status_logs (46956) has deleted/unused inode 47083. Clear<y>? yes Entry 'status_2012-11-26_12h14m53.csv.gz' in /var/log/status_logs (46956) has deleted/unused inode 47085. Clear<y>? yes Entry 'status_2012-11-26_15h06m49.csv' in /var/log/status_logs (46956) has deleted/unused inode 47088. Clear<y>? yes Entry 'status_2012-11-20_14h50m09.csv' in /var/log/status_logs (46956) has deleted/unused inode 47073. Clear<y>? yes Entry 'status_2012-11-20_14h55m32.csv' in /var/log/status_logs (46956) has deleted/unused inode 47074. Clear<y>? yes Entry 'status_2012-11-26_11h04m36.csv.gz' in /var/log/status_logs (46956) has deleted/unused inode 47078. Clear<y>? yes Entry 'status_2012-11-26_11h54m45.csv.gz' in /var/log/status_logs (46956) has deleted/unused inode 47082. Clear<y>? yes Entry 'status_2012-11-26_12h12m20.csv.gz' in /var/log/status_logs (46956) has deleted/unused inode 47084. Clear<y>? yes Entry 'status_2012-11-26_12h33m52.csv.gz' in /var/log/status_logs (46956) has deleted/unused inode 47086. Clear<y>? yes Entry 'status_2012-11-26_10h51m59.csv.gz' in /var/log/status_logs (46956) has deleted/unused inode 47077. Clear<y>? yes Entry 'status_2012-11-26_11h17m09.csv.gz' in /var/log/status_logs (46956) has deleted/unused inode 47079. Clear<y>? yes Entry 'status_2012-11-26_12h54m11.csv.gz' in /var/log/status_logs (46956) has deleted/unused inode 47087. Clear<y>? yes Pass 3: Checking directory connectivity '..' in /etc/network/run (46948) is <The NULL inode> (0), should be /etc/network (46953). Fix<y>? yes Couldn't fix parent of inode 46948: Couldn't find parent directory entry Pass 4: Checking reference counts Unattached inode 46945 Connect to /lost+found<y>? yes Inode 46945 ref count is 2, should be 1. Fix<y>? yes Inode 46953 ref count is 5, should be 4. Fix<y>? yes Pass 5: Checking group summary information Block bitmap differences: -(208264--208266) -(210062--210068) -(211343--211491) -(213241--213250) -(213344--213393) -213397 -(213457--213463) -(213516--213521) -(213628--213655) -(213683--213688) -(213709--213728) -(215265--215300) -(215346--215365) -(221541--221551) -(221696--221704) -227517 Fix<y>? yes Free blocks count wrong for group #6 (17247, counted=17611). Fix<y>? yes Free blocks count wrong (161691, counted=162055). Fix<y>? yes Inode bitmap differences: +(47089--47090) +47093 +47095 +(47097--47099) +(47101--47104) -(47219--47220) -47222 -47224 -47228 -47231 -(47347--47348) -47350 -47352 -47356 -47359 -(47457--47488) -47985 -47996 -(47999--48000) -48017 -(48027--48028) -(48030--48032) -48049 -(48059--48060) -(48062--48064) -48081 -(48091--48092) -(48094--48096) Fix<y>? yes Free inodes count wrong for group #6 (7608, counted=7624). Fix<y>? yes Free inodes count wrong (61919, counted=61935). Fix<y>? yes embeddedrootwrite: ***** FILE SYSTEM WAS MODIFIED ***** embeddedrootwrite: ********** WARNING: Filesystem still has errors ********** embeddedrootwrite: 657/62592 files (24.4% non-contiguous), 87882/249937 blocks Embedded-PC-failsafe:~# Embedded-PC-failsafe:~# e2fsck /dev/sda3 e2fsck 1.41.3 (12-Oct-2008) embeddedrootwrite contains a file system with errors, check forced. Pass 1: Checking inodes, blocks, and sizes Pass 2: Checking directory structure Directory entry for '.' in ... (46948) is big. Split<y>? yes Missing '..' in directory inode 46948. Fix<y>? yes Setting filetype for entry '..' in ... (46948) to 2. Pass 3: Checking directory connectivity '..' in /etc/network/run (46948) is <The NULL inode> (0), should be /etc/network (46953). Fix<y>? yes Pass 4: Checking reference counts Inode 2 ref count is 12, should be 13. Fix<y>? yes Pass 5: Checking group summary information embeddedrootwrite: ***** FILE SYSTEM WAS MODIFIED ***** embeddedrootwrite: 657/62592 files (24.4% non-contiguous), 87882/249937 blocks Embedded-PC-failsafe:~# Embedded-PC-failsafe:~# e2fsck /dev/sda3 e2fsck 1.41.3 (12-Oct-2008) embeddedrootwrite: clean, 657/62592 files, 87882/249937 blocks

  Read the article

• Best way to 'harden' embedded ext4 file server against unexpected loss of power?

  - by Jeremy Friesner

  Hi all, First, a little background: my company makes an audio streaming device that is a headless, rack-mounted Linux box with a couple of SSDs attached. Each SSD is formatted with ext4. The users can connect to the system using Samba/CIFS to upload new audio files or access existing ones. There is also custom software for streaming out audio over the network. This is all fine. The only problem is that the users are audio people, not computer people, and see the system as a 'black box', not as a computer. Which means that at the end of the day, they aren't going to ssh in to the box and enter "/sbin/shutdown -h"; they are just going to cut power to the rack and leave, and expect things to still work properly the next day. Since ext4 has journalling, journal checksumming, etc, this mostly works. The only time it doesn't work is when someone uploads a new file via Samba and then cuts power to the system before the uploaded data has been fully flushed to the disk. In that case, they come in the next day and find that their new file has been truncated or is missing entirely, and are unhappy. My question is, what is the best way to avoid this problem? Is there a way to get smbd to call "sync" at the end of every upload? (Performance on uploads isn't so important, since they only happen occasionally). Or is there a way to tell ext4 to automatically flush within a few seconds of any change to a file? (Again, performance can be sacrificed for safety here) Should I set a particular write-ordering mode, activate barriers, etc?

  Read the article

1