Home Columns FileSystems Using Strace to Examine Your IO

Using Strace to Examine Your IO

Details
Hits: 33904

Article Index

Page 3 of 3

Final Comments

I hope this article has given you a bit of an introduction on how to use strace to examine the IO pattern of your code. Notice that I didn't have to modify the code or rebuild it with a new library. This means you run an strace on an existing binary. So it's possible to examine the output pattern of your own codes or ISV codes for which you don't have source code.

This article is only an introduction. The output from a real code is much more extensive and complicated. Plus, you have to examine the strace output carefully to understand the IO patterns.

For example, you need to examine the strace.out files for IO functions in addition to open(), write(), read(), and close(). You want to examine if the code writes a little bit of data (maybe a few kilobytes), then uses lseek to move to a different part of the file, writes some more data, and so on. This IO pattern can be very difficult for file systems to optimize because they don't know how far the code lseeks so they have to assume that the data needs to be flushed to disk. Consequently the file system is writing a bunch of small files which is difficult for any distributed or parallel file system to do well.

Of course, there are IO patterns that some file systems can recognize and adapt to. For example, the code starts at the end of a file, lseeks backward a few megabytes, reads a few kilobytes, lseeks backwards, reads a few kilobytes, and so on. Some distributed or parallel file systems can recognize this IO pattern and perform some optimizations, such as caching the next few megabytes, to improve performance.

The key message of this article is that you need to examine the IO performance of your codes and you can do it without too much difficulty using strace. You may want to examine the IO pattern for several reasons.

  • To learn if IO is a performance bottleneck
  • To understand your IO pattern
  • To more critically examine distributed or parallel file system offerings
    • To just be a better cluster person

I didn't start this article with a rant about how little people know about the IO pattern of the code. But it's true. I've seen people who have no idea how much improving the IO performance will improve the performance of their code. There are also people who claim they know the IO pattern of their code and swear that they have to have a massive high-speed IO system for vastly improved IO performance. They sometimes even quote you huge IO throughputs that they just have to have. But, in actuality they have no idea what improved IO performance will do for their code performance. On the other hand, there are people who have a true understanding of the IO pattern of their code. I like those people.

I hope this article starts your on your way to better understanding the IO pattern of your code so that you don't become one of the knuckleheads I just described.

Processing the strace output files is not easy. In fact, a Perl or Python script that can take the output and produce something that looks readable or even a histogram of the IO performance would be very useful. I've looked on Google and I haven't found any yet, but I think there are a couple out there somewhere. If you find them or if you write one yourself, please contact me through Cluster Monkey or post it to the beowulf mailing list to help everyone else.



Jeff Layton is having way too much fun writing this article, proving that it's hard to keep a good geek down. When he's not creating havoc in his household, he can be found hanging out at the Fry's coffee shop (never during working hours) and admiring the shiny new CPUs that come in, and cringing when someone buys Microsoft Vista. The fund I started to buy some storage for myself is still open and taking donations.

Sidebar One: Example strace from MPI code 

15:12:54.806723 execve("/home/laytonj/TESTING/code1", ["/home/laytonj/TESTING/code1"], [/* 46 vars */]) = 0
15:12:54.835463 uname({sys="Linux", node="home8", ...}) = 0
15:12:54.835909 brk(0)                  = 0x8133000
15:12:54.836061 access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory)
15:12:54.836275 open("tls/i686/libpthread.so.0", O_RDONLY) = -1 ENOENT (No such file or directory)
15:12:54.836426 open("tls/libpthread.so.0", O_RDONLY) = -1 ENOENT (No such file or directory)
15:12:54.836553 open("i686/libpthread.so.0", O_RDONLY) = -1 ENOENT (No such file or directory)
15:12:54.836681 open("libpthread.so.0", O_RDONLY) = -1 ENOENT (No such file or directory)
15:12:54.836805 open("/home/laytonj/bin/mpich2-1.0.5p4-g95-old/lib/tls/i686/libpthread.so.0", O_RDONLY) = -1 ENOENT (No such file or directory)
15:12:54.837000 stat64("/home/laytonj/bin/mpich2-1.0.5p4-g95-old/lib/tls/i686", 0xbfa5807c) = -1 ENOENT (No such file or directory)
15:12:54.837150 open("/home/laytonj/bin/mpich2-1.0.5p4-g95-old/lib/tls/libpthread.so.0", O_RDONLY) = -1 ENOENT (No such file or directory)
15:12:54.837307 stat64("/home/laytonj/bin/mpich2-1.0.5p4-g95-old/lib/tls", 0xbfa5807c) = -1 ENOENT (No such file or directory)
15:12:54.837454 open("/home/laytonj/bin/mpich2-1.0.5p4-g95-old/lib/i686/libpthread.so.0", O_RDONLY) = -1 ENOENT (No such file or directory)
15:12:54.837611 stat64("/home/laytonj/bin/mpich2-1.0.5p4-g95-old/lib/i686", 0xbfa5807c) = -1 ENOENT (No such file or directory)
15:12:54.837757 open("/home/laytonj/bin/mpich2-1.0.5p4-g95-old/lib/libpthread.so.0", O_RDONLY) = -1 ENOENT (No such file or directory)
15:12:54.837912 stat64("/home/laytonj/bin/mpich2-1.0.5p4-g95-old/lib", {st_mode=S_IFDIR|0775, st_size=4096, ...}) = 0
15:12:54.838136 open("/etc/ld.so.cache", O_RDONLY) = 3
15:12:54.838267 fstat64(3, {st_mode=S_IFREG|0644, st_size=87369, ...}) = 0
15:12:54.838431 old_mmap(NULL, 87369, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7f44000
15:12:54.838592 close(3)                = 0
15:12:54.838713 open("/lib/tls/libpthread.so.0", O_RDONLY) = 3
15:12:54.838879 read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\20xz\000"..., 512) = 512
15:12:54.839046 fstat64(3, {st_mode=S_IFREG|0755, st_size=61492, ...}) = 0
15:12:54.839213 old_mmap(0x7a3000, 70108, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7a3000
15:12:54.839365 old_mmap(0x7b1000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0xd000) = 0x7b1000
15:12:54.839547 old_mmap(0x7b3000, 4572, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7b3000
15:12:54.839716 close(3)                = 0
15:12:54.839828 open("tls/i686/librt.so.1", O_RDONLY) = -1 ENOENT (No such file or directory)
15:12:54.839961 open("tls/librt.so.1", O_RDONLY) = -1 ENOENT (No such file or directory)
15:12:54.840085 open("i686/librt.so.1", O_RDONLY) = -1 ENOENT (No such file or directory)
15:12:54.840209 open("librt.so.1", O_RDONLY) = -1 ENOENT (No such file or directory)
15:12:54.840329 open("/home/laytonj/bin/mpich2-1.0.5p4-g95-old/lib/librt.so.1", O_RDONLY) = -1 ENOENT (No such file or directory)
15:12:54.840484 open("/lib/tls/librt.so.1", O_RDONLY) = 3
15:12:54.840622 read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\320`\206"..., 512) = 512
15:12:54.840770 fstat64(3, {st_mode=S_IFREG|0755, st_size=36560, ...}) = 0
15:12:54.841065 old_mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7f43000
15:12:54.841224 old_mmap(0x864000, 81656, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x864000
15:12:54.841381 old_mmap(0x86c000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x7000) = 0x86c000
15:12:54.841557 old_mmap(0x86e000, 40696, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x86e000
15:12:54.841724 close(3)                = 0
15:12:54.841837 open("tls/i686/libm.so.6", O_RDONLY) = -1 ENOENT (No such file or directory)
15:12:54.841972 open("tls/libm.so.6", O_RDONLY) = -1 ENOENT (No such file or directory)
15:12:54.842095 open("i686/libm.so.6", O_RDONLY) = -1 ENOENT (No such file or directory)
15:12:54.842262 open("libm.so.6", O_RDONLY) = -1 ENOENT (No such file or directory)
15:12:54.842386 open("/home/laytonj/bin/mpich2-1.0.5p4-g95-old/lib/libm.so.6", O_RDONLY) = -1 ENOENT (No such file or directory)
15:12:54.842547 open("/lib/tls/libm.so.6", O_RDONLY) = 3
15:12:54.842799 read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\0ce\000"..., 512) = 512
15:12:54.842955 fstat64(3, {st_mode=S_IFREG|0755, st_size=138684, ...}) = 0
15:12:54.843123 old_mmap(0x653000, 139424, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x653000
15:12:54.843274 old_mmap(0x674000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x20000) = 0x674000
15:12:54.843443 close(3)                = 0
15:12:54.843562 open("tls/i686/libc.so.6", O_RDONLY) = -1 ENOENT (No such file or directory)
15:12:54.843698 open("tls/libc.so.6", O_RDONLY) = -1 ENOENT (No such file or directory)
15:12:54.843822 open("i686/libc.so.6", O_RDONLY) = -1 ENOENT (No such file or directory)
15:12:54.843945 open("libc.so.6", O_RDONLY) = -1 ENOENT (No such file or directory)
15:12:54.844067 open("/home/laytonj/bin/mpich2-1.0.5p4-g95-old/lib/libc.so.6", O_RDONLY) = -1 ENOENT (No such file or directory)
15:12:54.844228 open("/lib/tls/libc.so.6", O_RDONLY) = 3
15:12:54.844372 read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\20_\27"..., 512) = 512
15:12:54.844519 fstat64(3, {st_mode=S_IFREG|0755, st_size=1212936, ...}) = 0
15:12:54.844688 old_mmap(0x161000, 1215644, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x161000
15:12:54.844841 old_mmap(0x284000, 16384, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x123000) = 0x284000
15:12:54.845871 old_mmap(0x288000, 7324, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x288000
15:12:54.846073 close(3)                = 0
15:12:54.846237 old_mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7f42000
15:12:54.846647 mprotect(0x284000, 4096, PROT_READ) = 0
15:12:54.846962 mprotect(0x15d000, 4096, PROT_READ) = 0
15:12:54.847139 set_thread_area({entry_number:-1 -> 6, base_addr:0xb7f426c0, limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0, useable:1}) = 0
15:12:54.847271 munmap(0xb7f44000, 87369) = 0
15:12:54.847393 set_tid_address(0xb7f42708) = 5220
15:12:54.847515 rt_sigaction(SIGRTMIN, {0x7a7380, [], SA_SIGINFO}, NULL, 8) = 0
15:12:54.847672 rt_sigaction(SIGRT_1, {0x7a73f0, [], SA_RESTART|SA_SIGINFO}, NULL, 8) = 0
15:12:54.847802 rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0
15:12:54.847928 getrlimit(RLIMIT_STACK, {rlim_cur=8192*1024, rlim_max=RLIM_INFINITY}) = 0
15:12:54.848081 _sysctl({{CTL_KERN, KERN_VERSION}, 2, 0xbfa5880c, 30, (nil), 0}) = 0
15:12:54.848431 brk(0)                  = 0x8133000
15:12:54.848629 brk(0x8134000)          = 0x8134000
15:12:54.848744 brk(0x8138000)          = 0x8138000
15:12:54.848876 ioctl(0, SNDCTL_TMR_TIMEBASE or TCGETS, 0xbfa58938) = -1 EINVAL (Invalid argument)
15:12:54.849084 brk(0x813c000)          = 0x813c000
15:12:54.849221 ioctl(1, SNDCTL_TMR_TIMEBASE or TCGETS, 0xbfa58938) = -1 EINVAL (Invalid argument)
15:12:54.849349 brk(0x8140000)          = 0x8140000
15:12:54.849471 ioctl(2, SNDCTL_TMR_TIMEBASE or TCGETS, 0xbfa58938) = -1 EINVAL (Invalid argument)
15:12:54.849614 rt_sigaction(SIGFPE, {0x809a9b4, [FPE], SA_SIGINFO}, NULL, 8) = 0
15:12:54.849763 rt_sigaction(SIGHUP, {SIG_DFL}, NULL, 8) = 0
15:12:54.849882 rt_sigaction(SIGINT, {SIG_DFL}, NULL, 8) = 0
15:12:54.849999 rt_sigaction(SIGQUIT, {0x80ad77c, [], 0}, NULL, 8) = 0
15:12:54.850124 rt_sigaction(SIGALRM, {0x80ad77c, [], 0}, NULL, 8) = 0
15:12:54.850268 mmap2(NULL, 401408, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7ee0000
15:12:54.850543 gettimeofday({1189365174, 850723}, NULL) = 0
15:12:54.850791 open("/etc/resolv.conf", O_RDONLY) = 3
15:12:54.850954 fstat64(3, {st_mode=S_IFREG|0644, st_size=79, ...}) = 0
15:12:54.851127 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7f59000
15:12:54.851243 read(3, "nameserver 24.197.160.18\nnameser"..., 4096) = 79
15:12:54.851455 read(3, "", 4096)       = 0
15:12:54.851574 close(3)                = 0
15:12:54.851678 munmap(0xb7f59000, 4096) = 0
15:12:54.851795 uname({sys="Linux", node="home8", ...}) = 0
15:12:54.852132 socket(PF_INET, SOCK_STREAM, IPPROTO_IP) = 3
15:12:54.852296 setsockopt(3, SOL_TCP, TCP_NODELAY, [1], 4) = 0
15:12:54.852432 connect(3, {sa_family=AF_INET, sin_port=htons(52320), sin_addr=inet_addr("192.168.0.8")}, 16) = 0
15:12:54.853324 write(3, "cmd=init pmi_version=1 pmi_subve"..., 40) = 40
15:12:54.858664 read(3, "cmd=response_to_init pmi_version"..., 1023) = 57
15:12:54.858894 write(3, "cmd=get_maxes\n", 14) = 14
15:12:54.859677 read(3, "cmd=maxes kvsname_max=4096 keyle"..., 1023) = 59
15:12:54.859872 write(3, "cmd=get_appnum\n", 15) = 15
15:12:54.860576 read(3, "cmd=appnum appnum=0\n", 1023) = 20
15:12:54.860759 brk(0x8141000)          = 0x8141000
15:12:54.860895 write(3, "cmd=get_my_kvsname\n", 19) = 19
15:12:54.861660 read(3, "cmd=my_kvsname kvsname=kvs_home8"..., 1023) = 43
15:12:54.861857 brk(0x8142000)          = 0x8142000
15:12:54.861996 write(3, "cmd=get_my_kvsname\n", 19) = 19
15:12:54.862718 read(3, "cmd=my_kvsname kvsname=kvs_home8"..., 1023) = 43
15:12:54.862927 brk(0x8143000)          = 0x8143000
15:12:54.863084 socket(PF_INET, SOCK_STREAM, IPPROTO_IP) = 4
15:12:54.863236 fcntl64(4, F_GETFL)     = 0x2 (flags O_RDWR)
15:12:54.863350 fcntl64(4, F_SETFL, O_RDWR|O_NONBLOCK) = 0
15:12:54.863470 bind(4, {sa_family=AF_INET, sin_port=htons(0), sin_addr=inet_addr("0.0.0.0")}, 16) = 0
15:12:54.863621 listen(4, 128)          = 0
15:12:54.863775 getsockname(4, {sa_family=AF_INET, sin_port=htons(46099), sin_addr=inet_addr("0.0.0.0")}, [16]) = 0
15:12:54.863975 brk(0x8144000)          = 0x8144000
15:12:54.864101 brk(0x8145000)          = 0x8145000
15:12:54.864265 write(3, "cmd=put kvsname=kvs_home8_39119_"..., 117) = 117
15:12:54.866026 read(3, "cmd=put_result rc=0\n", 1023) = 20
15:12:54.866212 write(3, "cmd=barrier_in\n", 15) = 15
15:12:54.867231 read(3, "cmd=barrier_out\n", 1023) = 16
15:12:54.920090 rt_sigprocmask(SIG_BLOCK, [QUIT ALRM], NULL, 8) = 0
15:12:54.920283 rt_sigprocmask(SIG_UNBLOCK, [QUIT ALRM], NULL, 8) = 0
15:12:54.920349 rt_sigprocmask(SIG_BLOCK, [QUIT ALRM], NULL, 8) = 0
15:12:54.920416 rt_sigprocmask(SIG_UNBLOCK, [QUIT ALRM], NULL, 8) = 0
15:12:54.920483 rt_sigprocmask(SIG_BLOCK, [QUIT ALRM], NULL, 8) = 0
15:12:54.920557 access("testfile1", F_OK) = -1 ENOENT (No such file or directory)
15:12:54.920631 access(".", R_OK)       = 0
15:12:54.920687 access(".", W_OK)       = 0
15:12:54.920748 stat64("testfile1", 0xbfa56800) = -1 ENOENT (No such file or directory)
15:12:54.920816 open("testfile1", O_RDWR|O_CREAT|O_LARGEFILE, 0666) = 7
15:12:54.920956 brk(0x8149000)          = 0x8149000
15:12:54.921032 ioctl(7, SNDCTL_TMR_TIMEBASE or TCGETS, 0xbfa57788) = -1 ENOTTY (Inappropriate ioctl for device)
15:12:54.921102 rt_sigprocmask(SIG_UNBLOCK, [QUIT ALRM], NULL, 8) = 0
15:12:54.921164 rt_sigprocmask(SIG_BLOCK, [QUIT ALRM], NULL, 8) = 0
15:12:54.921308 brk(0x814a000)          = 0x814a000
15:12:54.921448 brk(0x814c000)          = 0x814c000
15:12:54.921675 brk(0x8150000)          = 0x8150000
15:12:54.922074 brk(0x8158000)          = 0x8158000
15:12:54.922918 brk(0x8167000)          = 0x8167000
15:12:54.924345 brk(0x8187000)          = 0x8187000
15:12:54.927224 mmap2(NULL, 258048, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7ea1000
15:12:54.933745 mmap2(NULL, 516096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7e23000
15:12:54.934780 munmap(0xb7ea1000, 258048) = 0
15:12:54.940880 mmap2(NULL, 401408, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7dc1000
15:12:54.943471 write(7, "\200\32\6\0@$tH\200$tH\300$tH\0%tH@%tH\200%tH\300%tH"..., 400008) = 400008
15:12:54.945790 ftruncate64(7, 400008)  = 0
15:12:54.945888 _llseek(7, 0, [400008], SEEK_END) = 0
15:12:54.945954 ftruncate64(7, 400008)  = 0
15:12:54.946010 _llseek(7, 0, [400008], SEEK_END) = 0
15:12:54.946069 rt_sigprocmask(SIG_UNBLOCK, [QUIT ALRM], NULL, 8) = 0
15:12:54.946159 rt_sigprocmask(SIG_BLOCK, [QUIT ALRM], NULL, 8) = 0
15:12:54.946223 munmap(0xb7dc1000, 401408) = 0
15:12:54.946316 close(7)                = 0
15:12:54.946374 munmap(0xb7e23000, 516096) = 0
15:12:54.946460 brk(0x8145000)          = 0x8145000
15:12:54.946540 rt_sigprocmask(SIG_UNBLOCK, [QUIT ALRM], NULL, 8) = 0
15:12:54.946641 write(3, "cmd=get kvsname=kvs_home8_39119_"..., 56) = 56
15:12:54.948536 read(3, "cmd=get_result rc=0 value=port#4"..., 1023) = 81
15:12:54.948696 brk(0x8146000)          = 0x8146000
15:12:54.948774 socket(PF_INET, SOCK_STREAM, IPPROTO_IP) = 7
15:12:54.948852 fcntl64(7, F_GETFL)     = 0x2 (flags O_RDWR)
15:12:54.948910 fcntl64(7, F_SETFL, O_RDWR|O_NONBLOCK) = 0
15:12:54.948994 setsockopt(7, SOL_TCP, TCP_NODELAY, [1], 4) = 0
15:12:54.949061 connect(7, {sa_family=AF_INET, sin_port=htons(49392), sin_addr=inet_addr("192.168.0.8")}, 16) = -1 EINPROGRESS (Operation now in progress)
15:12:54.949235 poll([{fd=4, events=POLLIN, revents=POLLIN}, {fd=7, events=POLLOUT, revents=POLLOUT}], 2, -1) = 2
15:12:54.949323 getpeername(7, {sa_family=AF_INET, sin_port=htons(49392), sin_addr=inet_addr("192.168.0.8")}, [16]) = 0
15:12:54.949526 brk(0x8147000)          = 0x8147000
15:12:54.949596 accept(4, {sa_family=AF_INET, sin_port=htons(43533), sin_addr=inet_addr("192.168.0.8")}, [16]) = 8
15:12:54.949699 fcntl64(8, F_GETFL)     = 0x2 (flags O_RDWR)
15:12:54.949752 fcntl64(8, F_SETFL, O_RDWR|O_NONBLOCK) = 0
15:12:54.949808 setsockopt(8, SOL_TCP, TCP_NODELAY, [1], 4) = 0
15:12:54.949871 poll([{fd=4, events=POLLIN}, {fd=7, events=POLLOUT, revents=POLLOUT}, {fd=8, events=POLLIN, revents=POLLIN}], 3, -1) = 2
15:12:54.949946 writev(7, [{"\27\0\0\0\24\0\0\0\1\0\0\0\0\0\0\0\210\32\6\0\0\0\0\0\210"..., 32}, {"kvs_home8_39119_5_0\0", 20}], 2) = 52
15:12:54.950066 read(8, "\27\0\0\0\24\0\0\0\0\0\0\0\0\0\0\0\210\32\6\0\0\0\0\0\210"..., 32) = 32
15:12:54.950149 poll([{fd=4, events=POLLIN}, {fd=7, events=POLLIN}, {fd=8, events=POLLIN, revents=POLLIN}], 3, -1) = 1
15:12:54.950217 read(8, "kvs_home8_39119_5_0\0", 20) = 20
15:12:54.950283 poll([{fd=4, events=POLLIN}, {fd=7, events=POLLIN}, {fd=8, events=POLLOUT, revents=POLLOUT}], 3, -1) = 1
15:12:54.950350 write(8, "\31\0\0\0\1\0\0\0\0\0\0\0\0\0\0\0\210\32\6\0\0\0\0\0\210"..., 32) = 32
15:12:54.950766 poll([{fd=4, events=POLLIN}, {fd=7, events=POLLIN}, {fd=8, events=POLLIN, revents=POLLIN}], 3, -1) = 1
15:12:54.950834 read(8, "\0\0\0\0\1\0\0\0\0\0\1\0\1\0\0\254\0\0\0\0\1\0\0\0\1\0"..., 32) = 32
15:12:54.950916 poll([{fd=4, events=POLLIN, revents=POLLIN}, {fd=7, events=POLLIN}, {fd=8, events=POLLIN}], 3, -1) = 1
15:12:55.026209 brk(0x8148000)          = 0x8148000
15:12:55.026365 accept(4, {sa_family=AF_INET, sin_port=htons(43538), sin_addr=inet_addr("192.168.0.8")}, [16]) = 10
15:12:55.026576 fcntl64(10, F_GETFL)    = 0x2 (flags O_RDWR)
15:12:55.026771 fcntl64(10, F_SETFL, O_RDWR|O_NONBLOCK) = 0
15:12:55.026915 setsockopt(10, SOL_TCP, TCP_NODELAY, [1], 4) = 0
15:12:55.026984 poll([{fd=4, events=POLLIN}, {fd=7, events=POLLIN}, {fd=8, events=POLLIN}, {fd=10, events=POLLIN, revents=POLLIN}], 4, -1) = 1
15:12:55.027057 read(10, "\27\0\0\0\24\0\0\0\3\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 32) = 32
15:12:55.027138 poll([{fd=4, events=POLLIN}, {fd=7, events=POLLIN}, {fd=8, events=POLLIN}, {fd=10, events=POLLIN, revents=POLLIN}], 4, -1) = 1
15:12:55.027209 read(10, "kvs_home8_39119_5_0\0", 20) = 20
15:12:55.027276 poll([{fd=4, events=POLLIN}, {fd=7, events=POLLIN}, {fd=8, events=POLLIN}, {fd=10, events=POLLOUT, revents=POLLOUT}], 4, -1) = 1
15:12:55.027347 write(10, "\31\0\0\0\1\0\0\0\3\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 32) = 32
15:12:55.027476 poll([{fd=4, events=POLLIN}, {fd=7, events=POLLIN}, {fd=8, events=POLLIN}, {fd=10, events=POLLIN, revents=POLLIN}], 4, -1) = 1
15:12:55.027824 read(10, "\0\0\0\0\1\0\0\0\3\0\1\0\1\0\0\254\0\0\0\0\1\0\0\0\1\0"..., 32) = 32
15:12:55.027909 poll([{fd=4, events=POLLIN}, {fd=7, events=POLLIN, revents=POLLIN}, {fd=8, events=POLLIN}, {fd=10, events=POLLIN}], 4, -1) = 1
15:12:55.028100 read(7, "\31\0\0\0\1\0\0\0\1\0\0\0\0\0\0\0\210\32\6\0\0\0\0\0\210"..., 32) = 32
15:12:55.028202 poll([{fd=4, events=POLLIN}, {fd=7, events=POLLIN|POLLOUT, revents=POLLOUT}, {fd=8, events=POLLIN}, {fd=10, events=POLLIN}], 4, -1) = 1
15:12:55.028276 writev(7, [{"\0\0\0\0\1\0\0\0\1\0\1\0\1\0\0\254\0\0\0\0\1\0\0\0\1\0"..., 32}], 1) = 32
15:12:55.028378 write(10, "\0\0\0\0\1\0\0\0\1\0\1\0\0\0\0\254\0\0\0\0\1\0\0\0\1\0"..., 32) = 32
15:12:55.028515 write(8, "\25\0\0\0\0\0\0\0\0008\24\10\360\254\17\10$\213\245\277"..., 32) = 32
15:12:55.028750 write(7, "\25\0\0\0\0\0\0\0\0008\24\10\360\254\17\10$\213\245\277"..., 32) = 32
15:12:55.029021 write(10, "\25\0\0\0\0\0\0\0\0008\24\10\360\254\17\10$\213\245\277"..., 32) = 32
15:12:55.029188 poll([{fd=4, events=POLLIN}, {fd=7, events=POLLIN}, {fd=8, events=POLLIN}, {fd=10, events=POLLIN, revents=POLLIN}], 4, -1) = 1
15:12:55.029449 read(10, "\25\0\0\0\0\0\0\0\0008\24\10\360\254\17\10t\223\354\277"..., 32) = 32
15:12:55.029603 write(10, "\25\0\0\0\1\0\0\0\315\343!\0\1\0\0\0\0\0\0\0\0\0\0\0hP"..., 32) = 32
15:12:55.029763 poll([{fd=4, events=POLLIN}, {fd=7, events=POLLIN}, {fd=8, events=POLLIN}, {fd=10, events=POLLIN, revents=POLLIN}], 4, -1) = 1
15:12:55.029955 read(10, "\25\0\0\0\1\0\0\0\315\343!\0\1\0\0\0\0\0\0\0\0\0\0\0hP"..., 32) = 32
15:12:55.030105 fcntl64(10, F_GETFL)    = 0x802 (flags O_RDWR|O_NONBLOCK)
15:12:55.030238 fcntl64(10, F_SETFL, O_RDWR) = 0
15:12:55.030343 close(10)               = 0
15:12:55.030473 poll([{fd=4, events=POLLIN}, {fd=7, events=POLLIN}, {fd=8, events=POLLIN, revents=POLLIN}], 3, -1) = 1
15:12:55.032021 read(8, "\25\0\0\0\1\0\0\0\0008\24\10\360\254\17\10dz\260\277\360"..., 32) = 32
15:12:55.032098 write(8, "\25\0\0\0\1\0\0\0\315\343!\0\1\0\0\0\0\0\0\0\0\0\0\0hP"..., 32) = 32
15:12:55.032178 fcntl64(8, F_GETFL)     = 0x802 (flags O_RDWR|O_NONBLOCK)
15:12:55.032233 fcntl64(8, F_SETFL, O_RDWR) = 0
15:12:55.032283 close(8)                = 0
15:12:55.032348 poll([{fd=4, events=POLLIN}, {fd=7, events=POLLIN, revents=POLLIN}], 2, -1) = 1
15:12:55.035104 read(7, "\25\0\0\0\1\0\0\0\0008\24\10\360\254\17\10tw\306\277\0"..., 32) = 32
15:12:55.035187 write(7, "\25\0\0\0\1\0\0\0\315\343!\0\1\0\0\0\0\0\0\0\0\0\0\0hP"..., 32) = 32
15:12:55.035271 fcntl64(7, F_GETFL)     = 0x802 (flags O_RDWR|O_NONBLOCK)
15:12:55.035326 fcntl64(7, F_SETFL, O_RDWR) = 0
15:12:55.035375 close(7)                = 0
15:12:55.035449 fcntl64(4, F_GETFL)     = 0x802 (flags O_RDWR|O_NONBLOCK)
15:12:55.035502 fcntl64(4, F_SETFL, O_RDWR) = 0
15:12:55.035551 close(4)                = 0
15:12:55.035626 write(3, "cmd=finalize\n", 13) = 13
15:12:55.036095 read(3, "cmd=finalize_ack\n", 1023) = 17
15:12:55.036191 shutdown(3, 2 /* send and receive */) = 0
15:12:55.036434 close(3)                = 0
15:12:55.036513 munmap(0xb7ee0000, 401408) = 0
15:12:55.036625 write(1, "MYRANK:1\n", 9) = 9
15:12:55.037358 exit_group(0)           = ?

You have no rights to post comments

Main Menu

Search

Login And Newsletter

Create an account to access exclusive content, comment on articles, and receive our newsletters.

Feedburner

feed-image Front Page RSS Feed

This work is licensed under CC BY-NC-SA 4.0

©2005-2023 Copyright Seagrove LLC, Some rights reserved. Except where otherwise noted, this site is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International. The Cluster Monkey Logo and Monkey Character are Trademarks of Seagrove LLC.