|
ru.unix.bsd
- RU.UNIX.BSD ------------------------------------------------------------------
From : Vadim Goncharov 2:5020/400 17 Apr 2006 00:20:28
To : Vadim Goncharov
Subject : Re: стянуть distfiles для всего дерева
--------------------------------------------------------------------------------
Hi Vadim Goncharov!
On Mon, 27 Feb 2006 16:09:43 +0000 (UTC); Vadim Goncharov wrote about 'стянуть
distfiles для всего дерева':
VG> Есть необходимость поддерживать локальное зеркало дистфайлов (все -дцать
VG> гигабайт) для почти полного дерева портов. Поскольку нужны не все
VG> категории (а также доступ в сеть ограниченный, не все дистфайлы удастся
VG> скачать), сделал скрипт - обходит каталоги портов и в каждом делает
Вот финальная версия скрипта. Умеет закачивать в несколько потоков
одновременно, исключать указанные порты из обработки и еще по мелочи
(конфигурябельно). Hадеюсь, держателям зеркал дистфайлов пригодится.
#!/bin/sh
#
# fetch distfiles for entire ports tree in several parallel downloads
# (c) nuclight, April 2006 (covered by BSD license)
# usage: nice -n 10 ./fetchdist.sh &
### you can customize script behaviour using variables below
# your ports directory
PORTSDIR=/usr/ports
# for which /usr/ports subdirs (categories) do distfile fetch (we don't
# need chinese etc. ports, as long as non-category subdirs in /usr/ports
CATEGORIES="accessibility archivers astro audio benchmarks biology cad comms
converters databases deskutils devel dns editors emulators finance ftp games
graphics irc japanese java lang mail math mbone misc multimedia net net-im
net-mgmt net-p2p news palm print russian science security shells sysutils
textproc ukrainian www x11 x11-clocks x11-fm x11-fonts x11-servers x11-themes
x11-toolkits x11-wm"
# do logging of all work, one logfile for each category, e.g.,
/tmp/fetchlog-audio
# specify here prefix part of logfile name for each category
LOG=/data/FreeBSD/fetchlog
# script can do parallel processing several categories, for speed
# specify here how many categories to fetch at the same time
MAXCHILDS=4
# WARNING! Some ports do unattended installation of other ports during only
# fetch stage, as they're required to fetch. This may be unacceptable for you.
# You can find such ports using command like this:
# find . -name 'Makefile*' -exec sh -c "grep FETCH_DEPENDS {} && echo ' (in
{})'" \;
# and then do something before fetch or exclude them in this script.
#
# You can exclude selected ports from fetching here by specifying egrep(1)
# pattern, e.g., EXCLUDE='category1/port1|category2/port2|category3/port3'
EXCLUDE='net/asterisk-devel'
# vars for fetch
export HTTP_TIMEOUT=30
export FTP_TIMEOUT=30
# vars for make
export BATCH=yes
export NO_IGNORE=yes
#export PACKAGE_BUILDING=yes
### customization end, actual script begins
oldpwd=`pwd`
cookie=fetchdcookie # we can't ps | grep <OurScriptName>, 'coz shell forks for
ps, grep, ...
cd $PORTSDIR || exit 1
sleep 1 # don't do too many forks/execs at the same time
# child process - we have a category as an argument
if [ "x$1" != "x" ]; then
if ! ls -1d $1/* | egrep $EXCLUDE > /dev/null; then
# exclude patterns not found, so we can
# safely do make on entire category subdir
cd $PORTSDIR/$1 || exit 1
make -k checksum > ${LOG}-$1 2>&1
else
lst=`ls -1pd $1/* | egrep -v $EXCLUDE | egrep '/$'`
for w in $lst; do
cd $PORTSDIR/$w
make checksum >> ${LOG}-$1 2>&1
done
fi
exit 0 # terminate child
fi
# main process, without arguments
for i in $CATEGORIES ; do
# wait 1 minute if we have already maximum of working childs
while [ `ps axww | grep $cookie | grep -v grep | wc -l` -ge $MAXCHILDS ]; do
sleep 60
done
cd $oldpwd && $0 $i $cookie & # start child in background for this category
done
--
WBR, Vadim Goncharov. ICQ#166852181 mailto:vadim_nuclight@mail.ru
[Moderator of RU.ANTI-ECOLOGY][FreeBSD][http://antigreen.org][LJ:/nuclight]
--- slrn/0.9.8.1 on FreeBSD 4.11/i386
* Origin: Nuclear Lightning @ Tomsk, TPU AVTF Hostel (2:5020/400@fidonet)
Вернуться к списку тем, сортированных по: возрастание даты уменьшение даты тема автор
|
