| Message ID | 20111202160833.GK5540@mudshark.cambridge.arm.com |
|---|---|
| State | New |
| Headers | show |
On Fri, Dec 02, 2011 at 04:08:33PM +0000, Will Deacon wrote: > Hi Russell, > > This is the same as the pull request I sent in this thread: > > http://lists.arm.linux.org.uk/lurker/message/20111201.132625.2e9e2d3e.en.html > > I thought I'd better send it as a separate post to highlight it as a PULL. Thanks. > The following changes since commit 2d13ccaa8797d7e599f3792aed4b1e44b47f94a5: > > Merge branch 'irqchip-consolidation' of git://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms into devel-stable (2011-11-21 21:56:56 +0000) As this is known to break some exynos stuff, I'd prefer not to have a pull request based on this merge - but on the following merge (I've just pushed out) to reduce the number of commits which are potentially problematical. Could you base this off my new devel-stable instead? The head commit is 53fadbdd83039bb1181e4ff76123d612cdf26c37.
Morning Russell, On Sat, Dec 03, 2011 at 09:14:28AM +0000, Russell King - ARM Linux wrote: > On Fri, Dec 02, 2011 at 04:08:33PM +0000, Will Deacon wrote: > > The following changes since commit 2d13ccaa8797d7e599f3792aed4b1e44b47f94a5: > > > > Merge branch 'irqchip-consolidation' of git://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms into devel-stable (2011-11-21 21:56:56 +0000) > > As this is known to break some exynos stuff, I'd prefer not to have a pull > request based on this merge - but on the following merge (I've just pushed > out) to reduce the number of commits which are potentially problematical. > > Could you base this off my new devel-stable instead? The head commit > is 53fadbdd83039bb1181e4ff76123d612cdf26c37. No problem, I've pushed out the new branch and you should see it once the mirror syncs. Cheers, Will
On Fri, Dec 02, 2011 at 04:08:33PM +0000, Will Deacon wrote: > The following changes since commit 2d13ccaa8797d7e599f3792aed4b1e44b47f94a5: > > Merge branch 'irqchip-consolidation' of git://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms into devel-stable (2011-11-21 21:56:56 +0000) As this is known to be broken on OMAP, could I have this based upon some other commit - maybe ba8bb18a03f which I've just committed into devel-stable (which is the second 'fix' patch for OMAP) please.
Hi Russell, On Mon, Dec 05, 2011 at 11:23:42PM +0000, Russell King - ARM Linux wrote: > On Fri, Dec 02, 2011 at 04:08:33PM +0000, Will Deacon wrote: > > The following changes since commit 2d13ccaa8797d7e599f3792aed4b1e44b47f94a5: > > > > Merge branch 'irqchip-consolidation' of git://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms into devel-stable (2011-11-21 21:56:56 +0000) > > As this is known to be broken on OMAP, could I have this based upon > some other commit - maybe ba8bb18a03f which I've just committed into > devel-stable (which is the second 'fix' patch for OMAP) please. I'm struggling to pull from you [1] at the moment, but I'll rebase this once I manage to get the updated devel-stable. Will [1] http://ftp.arm.linux.org.uk/pub/linux/arm/kernel/git-cur/linux-2.6-arm.git
On Tue, Dec 06, 2011 at 10:14:39AM +0000, Will Deacon wrote: > Hi Russell, > > On Mon, Dec 05, 2011 at 11:23:42PM +0000, Russell King - ARM Linux wrote: > > On Fri, Dec 02, 2011 at 04:08:33PM +0000, Will Deacon wrote: > > > The following changes since commit 2d13ccaa8797d7e599f3792aed4b1e44b47f94a5: > > > > > > Merge branch 'irqchip-consolidation' of git://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms into devel-stable (2011-11-21 21:56:56 +0000) > > > > As this is known to be broken on OMAP, could I have this based upon > > some other commit - maybe ba8bb18a03f which I've just committed into > > devel-stable (which is the second 'fix' patch for OMAP) please. > > I'm struggling to pull from you [1] at the moment, but I'll rebase this once I > manage to get the updated devel-stable. It looks like the machine is hitting the memory and disk io bandwidth limits. Unfortunately, git seems to have high demands for both of those, weighing in at around 300-400MB per pack process, and seemingly wanting to read all the pack files too.
On Tue, Dec 06, 2011 at 10:38:43AM +0000, Russell King - ARM Linux wrote: > > > As this is known to be broken on OMAP, could I have this based upon > > > some other commit - maybe ba8bb18a03f which I've just committed into > > > devel-stable (which is the second 'fix' patch for OMAP) please. > > > > I'm struggling to pull from you [1] at the moment, but I'll rebase this once I > > manage to get the updated devel-stable. > > It looks like the machine is hitting the memory and disk io bandwidth > limits. Unfortunately, git seems to have high demands for both of those, > weighing in at around 300-400MB per pack process, and seemingly wanting > to read all the pack files too. Ok, I've got it now and have pushed out the new branch (kexec/idmap) based on the commit that you suggested above. Cheers, Will
On Tue, 6 Dec 2011, Russell King - ARM Linux wrote: > On Tue, Dec 06, 2011 at 10:14:39AM +0000, Will Deacon wrote: > > Hi Russell, > > > > On Mon, Dec 05, 2011 at 11:23:42PM +0000, Russell King - ARM Linux wrote: > > > On Fri, Dec 02, 2011 at 04:08:33PM +0000, Will Deacon wrote: > > > > The following changes since commit 2d13ccaa8797d7e599f3792aed4b1e44b47f94a5: > > > > > > > > Merge branch 'irqchip-consolidation' of git://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms into devel-stable (2011-11-21 21:56:56 +0000) > > > > > > As this is known to be broken on OMAP, could I have this based upon > > > some other commit - maybe ba8bb18a03f which I've just committed into > > > devel-stable (which is the second 'fix' patch for OMAP) please. > > > > I'm struggling to pull from you [1] at the moment, but I'll rebase this once I > > manage to get the updated devel-stable. > > It looks like the machine is hitting the memory and disk io bandwidth > limits. Unfortunately, git seems to have high demands for both of those, > weighing in at around 300-400MB per pack process, and seemingly wanting > to read all the pack files too. Make sure the repo on that machine is nicely packed. Running "git gc" (gc as in garbage collect) once in a while is a good thing to do, especially that you now have the smart HTTP protocol enabled. That will bring the memory usage way down, and serving requests will be much faster too. It is safe to put that in a cron job once a week or so, even if concurrent requests are being serviced. Nicolas
On Tue, Dec 06, 2011 at 11:25:47PM -0500, Nicolas Pitre wrote: > Make sure the repo on that machine is nicely packed. Running "git gc" > (gc as in garbage collect) once in a while is a good thing to do, > especially that you now have the smart HTTP protocol enabled. That will > bring the memory usage way down, and serving requests will be much > faster too. It is safe to put that in a cron job once a week or so, > even if concurrent requests are being serviced. Well, I tried an experiment. On my laptop, if I run git fsck, it takes around about 20 minutes to complete. On ZenIV, I started this, this morning: $ GIT_DIR=linux-2.6-arm.git git fsck and it's now (this evening) some 10 hours after, its still going. This is the exact same repository (as it's an rsync'd copy of the git objects and packs which are on the laptop.) $ ps aux | grep git USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND rmk 26731 1.4 1.0 382716 22036 pts/71 D+ 08:37 9:13 git fsck apache 29810 1.0 10.0 793288 206564 ? Sl 09:24 6:09 git pack-objects --revs --all --stdout --progress --delta-base-offset apache 30923 1.0 10.0 790560 206260 ? Sl 09:39 5:53 git pack-objects --revs --all --stdout --progress --delta-base-offset So, it's consumed 9h of CPU time, the git pack-objects have taken some 5-6 hours... $ vmstat 10 procs ------------memory---------- ---swap-- -----io---- --system-- -----cpu----- r b swpd free buff cache si so bi bo in cs us sy id wa st 0 4 2502200 45568 8252 125912 50 16 37 62 1 1 1 1 91 7 0 0 4 2502068 46056 8176 126016 44 0 23858 9 2231 1216 1 4 1 94 0 0 4 2501724 45100 8188 126764 76 0 22160 6 2269 1196 1 4 4 91 0 $ iostat 10 ... avg-cpu: %user %nice %system %iowait %steal %idle 0.88 0.00 4.10 91.71 0.00 3.32 Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn sdf 1.20 10.40 4.80 104 48 sde 0.70 16.00 0.00 160 0 sdd 16.00 432.00 90.40 4320 904 sdg 126.00 12614.40 0.00 126144 0 md0 3.90 26.40 4.80 264 48 sda 139.90 12292.80 0.00 122928 0 sdb 136.10 12393.60 0.00 123936 0 sdc 113.70 12238.40 0.00 122384 0 md1 628.20 49643.20 0.00 496432 0 The repository resides on md1 (sdg, sda, sdb, sdc), the swap partially on sdd and partially on md0 (sdf and sde). If I kill all git processes, the disks fall back to about 2MB/s over 10 seconds of analysis. As you can see, git fsck seems to be pulling data at around 50MB/s, presumably for 9 hours - this is rediculous because there's only 500MB of git data for it to read! What are the top six users of system memory? rmk 26731 1.4 1.0 380600 21348 pts/71 D+ 08:37 9:17 git fsck named 30776 0.4 1.4 130804 29212 ? Ssl Oct08 386:55 /usr/sbin/named-sdb root 2475 0.6 2.0 58940 41376 ? S 10:40 3:38 spamd child exim 2148 0.0 4.4 160228 91100 ? Ssl Jun24 183:50 clamd.exim apache 29810 1.0 10.2 794312 210816 ? Sl 09:24 6:27 git pack-objects apache 30923 1.0 10.3 794656 213536 ? Sl 09:39 6:10 git pack-objects and lower down there's the bunch of apache httpd processes. What this is saying to me is that git can't run sensibly on a dual-core P4, 3GHz machine with 2G of RAM and 4G swap, with a disk IO subsystem capable of about 50MB/s - basically, git is driving ZenIV into the ground (and I believe git was also responsible for ZenIV having a load average hitting a few hundred several months ago which resulted in us having to have it rebooted.) Last bit of information - from /proc/29810/maps: 0012b000-00148000 r-xp 00000000 08:33 179255 /lib/ld-2.13.so 00148000-00149000 r--p 0001c000 08:33 179255 /lib/ld-2.13.so 00149000-0014a000 rw-p 0001d000 08:33 179255 /lib/ld-2.13.so 0014c000-002cf000 r-xp 00000000 08:33 179409 /lib/libc-2.13.so 002cf000-002d0000 ---p 00183000 08:33 179409 /lib/libc-2.13.so 002d0000-002d2000 r--p 00183000 08:33 179409 /lib/libc-2.13.so 002d2000-002d3000 rw-p 00185000 08:33 179409 /lib/libc-2.13.so 002d3000-002d6000 rw-p 00000000 00:00 0 00304000-0031b000 r-xp 00000000 08:33 179711 /lib/libpthread-2.13.so 0031b000-0031c000 r--p 00016000 08:33 179711 /lib/libpthread-2.13.so 0031c000-0031d000 rw-p 00017000 08:33 179711 /lib/libpthread-2.13.so 0031d000-0031f000 rw-p 00000000 00:00 0 00328000-0033c000 r-xp 00000000 08:33 179755 /lib/libz.so.1.2.5 0033c000-0033d000 rw-p 00013000 08:33 179755 /lib/libz.so.1.2.5 006da000-006db000 r-xp 00000000 00:00 0 [vdso] 08047000-08171000 r-xp 00000000 08:32 600804 /usr/libexec/git-core/git 08171000-08176000 rw-p 0012a000 08:32 600804 /usr/libexec/git-core/git 08176000-081be000 rw-p 00000000 00:00 0 0a06a000-0f1b0000 rw-p 00000000 00:00 0 [heap] 80a74000-90836000 rw-p 00000000 00:00 0 986bc000-9a6bc000 r--p 00000000 09:01 106316459 /archive/ftp/pub/linux/arm/kernel/git-cur/linux-2.6-arm.git/objects/pack/pack-895bab46ff2e2e3d88eb2ebb8919a86104e6f59e.pack 9a6bc000-9c6bc000 r--p 0b000000 09:01 106316459 /archive/ftp/pub/linux/arm/kernel/git-cur/linux-2.6-arm.git/objects/pack/pack-895bab46ff2e2e3d88eb2ebb8919a86104e6f59e.pack 9c6bc000-9e6bd000 rw-p 00000000 00:00 0 a0330000-a0331000 ---p 00000000 00:00 0 a0331000-a0d31000 rw-p 00000000 00:00 0 a0d31000-a0d32000 ---p 00000000 00:00 0 a0d32000-a1732000 rw-p 00000000 00:00 0 a1e72000-a3e72000 r--p 08000000 09:01 106316459 /archive/ftp/pub/linux/arm/kernel/git-cur/linux-2.6-arm.git/objects/pack/pack-895bab46ff2e2e3d88eb2ebb8919a86104e6f59e.pack a3e72000-a5e72000 r--p 12000000 09:01 106316459 /archive/ftp/pub/linux/arm/kernel/git-cur/linux-2.6-arm.git/objects/pack/pack-895bab46ff2e2e3d88eb2ebb8919a86104e6f59e.pack a5e72000-a7e72000 r--p 02000000 09:01 106316459 /archive/ftp/pub/linux/arm/kernel/git-cur/linux-2.6-arm.git/objects/pack/pack-895bab46ff2e2e3d88eb2ebb8919a86104e6f59e.pack a8732000-a9e72000 r--p 1a000000 09:01 106316459 /archive/ftp/pub/linux/arm/kernel/git-cur/linux-2.6-arm.git/objects/pack/pack-895bab46ff2e2e3d88eb2ebb8919a86104e6f59e.pack a9e72000-abe72000 r--p 05000000 09:01 106316459 /archive/ftp/pub/linux/arm/kernel/git-cur/linux-2.6-arm.git/objects/pack/pack-895bab46ff2e2e3d88eb2ebb8919a86104e6f59e.pack abe72000-ac6fd000 rw-p 00000000 00:00 0 ac6fd000-ae6fd000 r--p 06000000 09:01 106316459 /archive/ftp/pub/linux/arm/kernel/git-cur/linux-2.6-arm.git/objects/pack/pack-895bab46ff2e2e3d88eb2ebb8919a86104e6f59e.pack ae6fd000-af3fb000 rw-p 00000000 00:00 0 af946000-af947000 rw-p 00000000 00:00 0 afb00000-afbb3000 rw-p 00000000 00:00 0 afbb3000-afc00000 ---p 00000000 00:00 0 afcc1000-b1652000 rw-p 00000000 00:00 0 b1800000-b1900000 rw-p 00000000 00:00 0 b1ae8000-b1af0000 r--p 00000000 09:01 88555537 /archive/ftp/pub/linux/arm/kernel/git-cur/linux-2.6-arm.git/objects/pack/pack-60ee04b368ada02ba788d17a537c71bad7471a7a.idx b1af0000-b1af3000 r--p 00000000 09:01 106318826 /archive/ftp/pub/linux/arm/kernel/git-cur/linux-2.6-arm.git/objects/pack/pack-87f106a487c423809976c6868ec7f1e9fc12e676.idx b1af3000-b1af6000 r--p 00000000 09:01 88555528 /archive/ftp/pub/linux/arm/kernel/git-cur/linux-2.6-arm.git/objects/pack/pack-1a10fd87008d0e06d2ce83efff9bf4ba7abb12e8.idx b1af6000-b1b03000 r--p 00000000 09:01 88555566 /archive/ftp/pub/linux/arm/kernel/git-cur/linux-2.6-arm.git/objects/pack/pack-6ed1c32d2ec8e0cf20c9ef7d8b813e1f63f67203.idx b1d00000-b1de8000 rw-p 00000000 00:00 0 b1de8000-b1e00000 ---p 00000000 00:00 0 b1e00000-b1e51000 rw-p 00000000 00:00 0 b1e51000-b1f00000 ---p 00000000 00:00 0 b1f00000-b1ffe000 rw-p 00000000 00:00 0 b1ffe000-b2000000 ---p 00000000 00:00 0 b2000000-b20e5000 rw-p 00000000 00:00 0 b20e5000-b2100000 ---p 00000000 00:00 0 b2100000-b21fc000 rw-p 00000000 00:00 0 b21fc000-b2200000 ---p 00000000 00:00 0 b2200000-b22fd000 rw-p 00000000 00:00 0 b22fd000-b2300000 ---p 00000000 00:00 0 b2300000-b23d7000 rw-p 00000000 00:00 0 b23d7000-b2400000 ---p 00000000 00:00 0 b2400000-b24fd000 rw-p 00000000 00:00 0 b24fd000-b2500000 ---p 00000000 00:00 0 b2500000-b25ea000 rw-p 00000000 00:00 0 b25ea000-b2600000 ---p 00000000 00:00 0 b2600000-b26e4000 rw-p 00000000 00:00 0 b26e4000-b2700000 ---p 00000000 00:00 0 b2700000-b27ee000 rw-p 00000000 00:00 0 b27ee000-b2800000 ---p 00000000 00:00 0 b2800000-b28ec000 rw-p 00000000 00:00 0 b28ec000-b2900000 ---p 00000000 00:00 0 b2900000-b29f3000 rw-p 00000000 00:00 0 b29f3000-b2a00000 ---p 00000000 00:00 0 b2b00000-b2cfe000 rw-p 00000000 00:00 0 b2cfe000-b2d00000 ---p 00000000 00:00 0 b2d00000-b2dfd000 rw-p 00000000 00:00 0 b2dfd000-b2e00000 ---p 00000000 00:00 0 b2f00000-b2ff4000 rw-p 00000000 00:00 0 b2ff4000-b3000000 ---p 00000000 00:00 0 b3100000-b31fe000 rw-p 00000000 00:00 0 b31fe000-b3200000 ---p 00000000 00:00 0 b3300000-b33f1000 rw-p 00000000 00:00 0 b33f1000-b3400000 ---p 00000000 00:00 0 b3500000-b35f6000 rw-p 00000000 00:00 0 b35f6000-b3600000 ---p 00000000 00:00 0 b3600000-b36f3000 rw-p 00000000 00:00 0 b36f3000-b3700000 ---p 00000000 00:00 0 b3700000-b37f9000 rw-p 00000000 00:00 0 b37f9000-b3800000 ---p 00000000 00:00 0 b3900000-b39f3000 rw-p 00000000 00:00 0 b39f3000-b3a00000 ---p 00000000 00:00 0 b3b03000-b76ab000 r--p 00000000 09:01 106316443 /archive/ftp/pub/linux/arm/kernel/git-cur/linux-2.6-arm.git/objects/pack/pack-895bab46ff2e2e3d88eb2ebb8919a86104e6f59e.idx b76ab000-b76b1000 r--p 00000000 09:01 106316389 /archive/ftp/pub/linux/arm/kernel/git-cur/linux-2.6-arm.git/objects/pack/pack-742507456b81724d4b1b7c62a9c7b1a08dea788c.idx b76b1000-b76b3000 r--p 00000000 09:01 88555543 /archive/ftp/pub/linux/arm/kernel/git-cur/linux-2.6-arm.git/objects/pack/pack-01d8e6a80f43c483b170f252a63549c3f3177435.idx b76b3000-b76b5000 r--p 00000000 09:01 106316378 /archive/ftp/pub/linux/arm/kernel/git-cur/linux-2.6-arm.git/objects/pack/pack-22db475e3babd502f5462dbd53b0083f63054da8.idx b76b5000-b76b7000 r--p 00000000 09:01 106319097 /archive/ftp/pub/linux/arm/kernel/git-cur/linux-2.6-arm.git/objects/pack/pack-81e9156c16956b1ca13d0786f26c57da8a8c4d00.idx b76b7000-b76b9000 r--p 00000000 09:01 106316404 /archive/ftp/pub/linux/arm/kernel/git-cur/linux-2.6-arm.git/objects/pack/pack-02bd0e7dcc0d4f9ba1faef6cb90d3080451223ab.idx b76b9000-b76be000 r--p 00000000 09:01 106320451 /archive/ftp/pub/linux/arm/kernel/git-cur/linux-2.6-arm.git/objects/pack/pack-64a9ef29c0dd71c0ada102c21a6d3297715b5ec0.idx b76be000-b76c0000 r--p 00000000 09:01 88555532 /archive/ftp/pub/linux/arm/kernel/git-cur/linux-2.6-arm.git/objects/pack/pack-cf138203ddca32052979b5eadc8b03fa8d9f6313.idx b76c0000-b76c2000 r--p 00000000 09:01 88555522 /archive/ftp/pub/linux/arm/kernel/git-cur/linux-2.6-arm.git/objects/pack/pack-08c44c7f188425373f8ecb95e4bb342ee40d340e.idx b76c2000-b76cb000 r--p 00000000 09:01 88555534 /archive/ftp/pub/linux/arm/kernel/git-cur/linux-2.6-arm.git/objects/pack/pack-1fd3ce5f1145636ad725dec83f5ef91b80ec1cf3.idx b76cb000-b76cf000 r--p 00000000 09:01 88555545 /archive/ftp/pub/linux/arm/kernel/git-cur/linux-2.6-arm.git/objects/pack/pack-0e5768a969e7c4ef4f3406a129b1271971ee5342.idx b76cf000-b76d4000 r--p 00000000 09:01 88555547 /archive/ftp/pub/linux/arm/kernel/git-cur/linux-2.6-arm.git/objects/pack/pack-92592ad40e58a75d3d1b30a0a46e7ad63817637b.idx b76d4000-b76d6000 r--p 00000000 09:01 88555549 /archive/ftp/pub/linux/arm/kernel/git-cur/linux-2.6-arm.git/objects/pack/pack-5b26c4eb6fed3ce5b26e921aee548a7f3259a11e.idx b76d6000-b76d9000 r--p 00000000 09:01 88555551 /archive/ftp/pub/linux/arm/kernel/git-cur/linux-2.6-arm.git/objects/pack/pack-4a9b526f37540b18dd2ff4e97ec94fcb49c0aeb6.idx b76d9000-b76dd000 r--p 00000000 09:01 88555553 /archive/ftp/pub/linux/arm/kernel/git-cur/linux-2.6-arm.git/objects/pack/pack-1a7d19bf1f4aa63bb54b0f7e77f5df8889099e3f.idx b76dd000-b76e0000 r--p 00000000 09:01 88555558 /archive/ftp/pub/linux/arm/kernel/git-cur/linux-2.6-arm.git/objects/pack/pack-7da553217dc96aa89cdcf50fe45615143aa01089.idx b76e0000-b76e6000 r--p 00000000 09:01 88555555 /archive/ftp/pub/linux/arm/kernel/git-cur/linux-2.6-arm.git/objects/pack/pack-e795f1594ea70cca88a97ad23d2d62da06941038.idx b76e6000-b76ea000 r--p 00000000 09:01 88555560 /archive/ftp/pub/linux/arm/kernel/git-cur/linux-2.6-arm.git/objects/pack/pack-4c79da67fb0435e3a4a7440a714fb9d617995387.idx b76e6000-b76ea000 r--p 00000000 09:01 88555560 /archive/ftp/pub/linux/arm/kernel/git-cur/linux-2.6-arm.git/objects/pack/pack-4c79da67fb0435e3a4a7440a714fb9d617995387.idx b76ea000-b76ef000 r--p 00000000 09:01 88555563 /archive/ftp/pub/linux/arm/kernel/git-cur/linux-2.6-arm.git/objects/pack/pack-63e7f343a0aa5f8f6841c81211db00ce95adbe76.idx b76ef000-b76f1000 rw-p 00000000 00:00 0 b76f5000-b76fa000 r--p 00000000 09:01 88555565 /archive/ftp/pub/linux/arm/kernel/git-cur/linux-2.6-arm.git/objects/pack/pack-b37609403c59702e384e42a9df4bc16e2b3c1072.idx b76fa000-b76fc000 r--p 00000000 09:01 88555542 /archive/ftp/pub/linux/arm/kernel/git-cur/linux-2.6-arm.git/objects/pack/pack-23f0d9e30ada6f829f6743a395bcb6f229c1453b.idx b76fc000-b7700000 r--p 00000000 09:01 88555571 /archive/ftp/pub/linux/arm/kernel/git-cur/linux-2.6-arm.git/objects/pack/pack-d073eba6bd4143e405ec9db4172d4406fa08ace8.idx b7700000-b7702000 r--p 00000000 09:01 88555574 /archive/ftp/pub/linux/arm/kernel/git-cur/linux-2.6-arm.git/objects/pack/pack-b2481dc10cc72226ff396e864e4a6d7e2c26137d.idx b7702000-b7704000 r--p 00000000 09:01 88555577 /archive/ftp/pub/linux/arm/kernel/git-cur/linux-2.6-arm.git/objects/pack/pack-866787d1e38a3c7c9ac0ab7057935bf5970bf319.idx b7704000-b7705000 rw-p 00000000 00:00 0 bf8dc000-bf8fd000 rw-p 00000000 00:00 0 [stack] It's worth noting that Linus tree currently has 19 pack files, my tree has an additional 9 on that.
On Wed, 7 Dec 2011, Russell King - ARM Linux wrote: > On Tue, Dec 06, 2011 at 11:25:47PM -0500, Nicolas Pitre wrote: > > Make sure the repo on that machine is nicely packed. Running "git gc" > > (gc as in garbage collect) once in a while is a good thing to do, > > especially that you now have the smart HTTP protocol enabled. That will > > bring the memory usage way down, and serving requests will be much > > faster too. It is safe to put that in a cron job once a week or so, > > even if concurrent requests are being serviced. > > Well, I tried an experiment. On my laptop, if I run git fsck, it takes > around about 20 minutes to complete. This is a bit long but reasonable. > On ZenIV, I started this, this morning: > > $ GIT_DIR=linux-2.6-arm.git git fsck > > and it's now (this evening) some 10 hours after, its still going. This > is the exact same repository (as it's an rsync'd copy of the git objects > and packs which are on the laptop.) Ouch! No, this is no good. [...] > As you can see, git fsck seems to be pulling data at around 50MB/s, > presumably for 9 hours - this is rediculous because there's only 500MB > of git data for it to read! Well, of course the fsck process will keep a tree in memory of the relationship between all objects and so on. So it certainly has potential for eating lots of memory and pushing the system into swap. And I don't think that fsck was optimized to minimize seeks like pack-objects does. Of course this is not very representative of a typical git pull process though. Assuming that people are already updated to v3.2-rc1, you can simulate the effect on the server by running: echo "v3.2-rc1..for-next" | \ git pack-objects --progress --thin --revs --stdout > /dev/null and you really really don't want such an operation to ever touch swap space. > What this is saying to me is that git can't run sensibly on a dual-core > P4, 3GHz machine with 2G of RAM and 4G swap, with a disk IO subsystem > capable of about 50MB/s - basically, git is driving ZenIV into the ground > (and I believe git was also responsible for ZenIV having a load average > hitting a few hundred several months ago which resulted in us having to > have it rebooted.) Most likely, yes. The disk throughput shouldn't be such a problem, but the lack of RAM certainly is. And 2g of RAM to deal with a repository the size of the Linux kernel is making it tight. So forget my suggestion about packing the repository on the server since it certainly doesn't have enough RAM to do a descent job. You should consider packing it elsewhere on a biffier machine, and copying the resulting pack to ZenIV. Having a well packed repository is one of the best way to limit memory consumption when serving a repository, but to produce that pack you need quite some RAM in the first place. I'm sure many corporations involved with ARM would be more than welcome to sponsor some 64-bit hardware for this server and at least 8G of RAM, which is relatively cheap these days. > It's worth noting that Linus tree currently has 19 pack files, my tree > has an additional 9 on that. One design requirement for the pack file format is that it should be self sufficient when on disk i.e. no delta representation may refer to a base object outside of the pack it is found in. Of course the wire protocol is totally the opposite i.e. the packed data sent during a transfer is free of any object data the other end already has. So upon reception over the smart protocol, any packed data has to be "completed" with a copy of the locally found objects for the pack to be self contained. There is therefore some data redundancy that accumulates as the number of packs in a repository grows. Hence the need to garbage collect once in a while. Having 28 packs is normally not that huge a deal, but maybe the relatively small amount of RAM makes it more critical. Nicolas
On Wed, Dec 07, 2011 at 03:38:31PM -0500, Nicolas Pitre wrote: > Most likely, yes. The disk throughput shouldn't be such a problem, but > the lack of RAM certainly is. And 2g of RAM to deal with a repository > the size of the Linux kernel is making it tight. So how much RAM is now required - and will be required in the future - for running git? Given that my laptop also has 2G of RAM, it seems from what you're saying that's also a problem there - or will be soon.
On Wed, 7 Dec 2011, Russell King - ARM Linux wrote: > On Wed, Dec 07, 2011 at 03:38:31PM -0500, Nicolas Pitre wrote: > > Most likely, yes. The disk throughput shouldn't be such a problem, but > > the lack of RAM certainly is. And 2g of RAM to deal with a repository > > the size of the Linux kernel is making it tight. > > So how much RAM is now required - and will be required in the future - > for running git? Given that my laptop also has 2G of RAM, it seems > from what you're saying that's also a problem there - or will be soon. Well, this depends on what operation you typically use Git for of course. Normally, checkout, commit, log, merge, etc. are operations that don't need to walk the entire repository data and are less affected by the repository growth than the size of the checked-out tree. The other class of operations such as fsck, repack, gc, clone, push/pull/fetch are likely to see their memory footprint grow with the size of the repository. Fortunately the size of the Linux Git repository tends to grow slower than Moore's law. Nicolas
Hi Russell, This is the same as the pull request I sent in this thread: http://lists.arm.linux.org.uk/lurker/message/20111201.132625.2e9e2d3e.en.html I thought I'd better send it as a separate post to highlight it as a PULL. Cheers, Will The following changes since commit 2d13ccaa8797d7e599f3792aed4b1e44b47f94a5: Merge branch 'irqchip-consolidation' of git://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms into devel-stable (2011-11-21 21:56:56 +0000) are available in the git repository at: git://git.kernel.org/pub/scm/linux/kernel/git/will/linux.git kexec/idmap Will Deacon (6): ARM: idmap: populate identity map pgd at init time using .init.text ARM: suspend: use idmap_pgd instead of suspend_pgd ARM: proc-*.S: place cpu_reset functions into .idmap.text section ARM: idmap: use idmap_pgd when setting up mm for reboot ARM: head.S: only include __turn_mmu_on in the initial identity mapping ARM: SMP: use idmap_pgd for mapping MMU enable during secondary booting arch/arm/include/asm/idmap.h | 14 +++++++++ arch/arm/include/asm/pgtable.h | 3 -- arch/arm/kernel/head.S | 18 ++++++----- arch/arm/kernel/sleep.S | 2 + arch/arm/kernel/smp.c | 32 +------------------- arch/arm/kernel/suspend.c | 18 ++--------- arch/arm/kernel/vmlinux.lds.S | 7 ++++ arch/arm/mm/idmap.c | 63 +++++++++++++++++++--------------------- arch/arm/mm/proc-arm1020.S | 3 ++ arch/arm/mm/proc-arm1020e.S | 3 ++ arch/arm/mm/proc-arm1022.S | 3 ++ arch/arm/mm/proc-arm1026.S | 3 ++ arch/arm/mm/proc-arm6_7.S | 4 ++ arch/arm/mm/proc-arm720.S | 3 ++ arch/arm/mm/proc-arm740.S | 3 ++ arch/arm/mm/proc-arm7tdmi.S | 3 ++ arch/arm/mm/proc-arm920.S | 3 ++ arch/arm/mm/proc-arm922.S | 3 ++ arch/arm/mm/proc-arm925.S | 3 ++ arch/arm/mm/proc-arm926.S | 3 ++ arch/arm/mm/proc-arm940.S | 3 ++ arch/arm/mm/proc-arm946.S | 3 ++ arch/arm/mm/proc-arm9tdmi.S | 3 ++ arch/arm/mm/proc-fa526.S | 3 ++ arch/arm/mm/proc-feroceon.S | 3 ++ arch/arm/mm/proc-mohawk.S | 3 ++ arch/arm/mm/proc-sa110.S | 3 ++ arch/arm/mm/proc-sa1100.S | 3 ++ arch/arm/mm/proc-v6.S | 3 ++ arch/arm/mm/proc-v7.S | 2 + arch/arm/mm/proc-xsc3.S | 3 ++ arch/arm/mm/proc-xscale.S | 3 ++ 32 files changed, 140 insertions(+), 89 deletions(-) create mode 100644 arch/arm/include/asm/idmap.h