[kwlug-disc] bash subprocesses & traps
John Steel
john at jskw.ca
Sun Jul 28 13:56:59 EDT 2024
Progress! I liked your idea of having the loop finish without needing to be killed. I switched to signup and sha256sum. Removing the PID file from the trap seems to gracefully stop the loop. I still think I don’t have a good mental picture of why this works and the previous one didn't
$ docker run -it -v $PWD:/app alpine sh -xc '
apk add bash procps
export FIFO_FILE="$(mktemp -u -p /dev/shm fifo_$$.XXXXXX)"
bash -x /app/checksum.sh &
sleep 0.2
jobs
ps -ef --forest
echo "First data to hash" > $FIFO_FILE
sleep 0.2
ps -ef --forest
echo "More data to hash" > $FIFO_FILE
sleep 1
echo "More data to hash" > $FIFO_FILE
sleep 1
echo "More data to hash" > $FIFO_FILE
kill -s SIGHUP %1
sleep 1
'
+ apk add bash procps
fetch https://dl-cdn.alpinelinux.org/alpine/v3.19/main/aarch64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.19/community/aarch64/APKINDEX.tar.gz
(1/7) Installing ncurses-terminfo-base (6.4_p20231125-r0)
(2/7) Installing libncursesw (6.4_p20231125-r0)
(3/7) Installing readline (8.2.1-r2)
(4/7) Installing bash (5.2.21-r0)
Executing bash-5.2.21-r0.post-install
(5/7) Installing libintl (0.22.3-r0)
(6/7) Installing libproc2 (4.0.4-r0)
(7/7) Installing procps-ng (4.0.4-r0)
Executing busybox-1.36.1-r15.trigger
OK: 13 MiB in 22 packages
+ mktemp -u -p /dev/shm fifo_1.XXXXXX
+ export 'FIFO_FILE=/dev/shm/fifo_1.DGoldG'
+ sleep 0.2
+ bash -x /app/checksum.sh
+ FIFO_FILE=/dev/shm/fifo_1.DGoldG
++ mktemp -p /dev/shm 13_cat_pid.XXXXXX
+ CAT_PID_FILE=/dev/shm/13_cat_pid.DOBkfG
+ export CAT_PID_FILE FIFO_FILE
+ mkfifo /dev/shm/fifo_1.DGoldG
+ trap finalize SIGTERM
+ trap finalize SIGHUP
+ trap finalize SIGINT
+ trap finalize SIGQUIT
+ wait
+ + '[' tee -f /dev/stderr/dev/shm/13_cat_pid.DOBkfG
']'
+ sha256sum
+ cat_pid=20
+ echo 20
+ wait 20
+ cat /dev/shm/fifo_1.DGoldG
+ jobs
[1]+ Running
+ ps -ef --forest
UID PID PPID C STIME TTY TIME CMD
root 1 0 0 17:48 pts/0 00:00:00 sh -xc apk add bash procps export FIFO_FILE="$(mktemp -u -p /dev/shm fifo_$$.XXXXXX)" bash
root 13 1 0 17:48 pts/0 00:00:00 bash -x /app/checksum.sh
root 18 13 0 17:48 pts/0 00:00:00 \_ tee /dev/stderr
root 17 13 0 17:48 pts/0 00:00:00 \_ bash -x /app/checksum.sh
root 20 17 0 17:48 pts/0 00:00:00 | \_ cat /dev/shm/fifo_1.DGoldG
root 19 13 0 17:48 pts/0 00:00:00 \_ sha256sum
root 21 1 0 17:48 pts/0 00:00:00 ps -ef --forest
+ echo 'First data to hash'
+ sleep 0.2
First data to hash
+ sleep 0.01
+ '[' -f /dev/shm/13_cat_pid.DOBkfG ']'
+ cat_pid=24
+ echo 24
+ wait 24
+ cat /dev/shm/fifo_1.DGoldG
+ ps -ef --forest
UID PID PPID C STIME TTY TIME CMD
root 1 0 0 17:48 pts/0 00:00:00 sh -xc apk add bash procps export FIFO_FILE="$(mktemp -u -p /dev/shm fifo_$$.XXXXXX)" bash
root 13 1 0 17:48 pts/0 00:00:00 bash -x /app/checksum.sh
root 18 13 0 17:48 pts/0 00:00:00 \_ tee /dev/stderr
root 17 13 0 17:48 pts/0 00:00:00 \_ bash -x /app/checksum.sh
root 24 17 0 17:48 pts/0 00:00:00 | \_ cat /dev/shm/fifo_1.DGoldG
root 19 13 0 17:48 pts/0 00:00:00 \_ sha256sum
root 25 1 0 17:48 pts/0 00:00:00 ps -ef --forest
+ echo 'More data to hash'
+ sleep 1
More data to hash
+ sleep 0.01
+ '[' -f /dev/shm/13_cat_pid.DOBkfG ']'
+ cat_pid=28
+ echo 28
+ wait 28
+ cat /dev/shm/fifo_1.DGoldG
+ echo 'More data to hash'
+ sleep 1
More data to hash
+ sleep 0.01
+ '[' -f /dev/shm/13_cat_pid.DOBkfG ']'
+ cat_pid=31
+ echo 31
+ + wait 31
cat /dev/shm/fifo_1.DGoldG
+ echo 'More data to hash'
+ kill -s SIGHUP '%1'
+ sleep 1
More data to hash
++ + finalizesleep
0.01
++ pgrep -fl 'cat fifo /dev/shm/fifo_1.DGoldG'
++ pgrep -fl sha256sum
19 sha256sum
+++ cat /dev/shm/13_cat_pid.DOBkfG
++ cat_pid=31
++ rm /dev/shm/13_cat_pid.DOBkfG
++ exit 0
+ '[' -f /dev/shm/13_cat_pid.DOBkfG ']'
23d54b3983757deb3a3f775175a36a236bb82badca0e6c79423a8cc059c9059b -
$ echo 'First data to hash
quote> More data to hash
quote> More data to hash
quote> More data to hash' | sha256sum
23d54b3983757deb3a3f775175a36a236bb82badca0e6c79423a8cc059c9059b -
> On Jul 28, 2024, at 1:28 PM, John Steel via kwlug-disc <kwlug-disc at kwlug.org> wrote:
>
> Oh! That's a neat idea, will try it out but I still want to understand what I'm doing wrong with my trap.
>
> I think my wait on cat is causing a problem. Maybe I should be putting my whole while loop into a subshell?
>
> On Sun, Jul 28, 2024, at 12:56 PM, Chris Frey wrote:
>> I'd recommend using some other mechanism of "end of stream" rather
>> than killing your script, since, as you found out, it halts all processing.
>>
>> Perhaps instead of kill, rename the file:
>>
>> echo "More data to hash" > $FIFO_FILE
>> # rename it so while loop exits
>> mv "$FIFIO_FILE" "$FIFO_FILE-finished"
>> # send empty data to trigger a fifo pipe close on the waiting cat
>> cat /dev/null > "$FIFO_FILE-finished"
>>
>> - Chris
>>
>>
>> On Sun, Jul 28, 2024 at 12:18:43PM -0400, John Steel via kwlug-disc wrote:
>> > I’m having trouble figuring out how to manage a subprocess… I’m probably overcomplicating this…
>> >
>> > I’ve made a script called checksum.sh. I want it to keep moving data from a fifo to sha256sum until it’s terminated. But it seems when I kill it the cat and sha256sum are gone before my finalize method is called? If I can kill cat and let that sub shell exit gracefully I think I should see the sha256sum come out on stderr.
>> >
>> > #!/bin/bash
>> > # This is checksum.sh
>> >
>> > FIFO_FILE="${FIFO_FILE:-sha256sum_fifo}"
>> > CAT_PID_FILE="$(mktemp -p /dev/shm $$_cat_pid.XXXXXX)"
>> > export CAT_PID_FILE FIFO_FILE
>> > mkfifo "$FIFO_FILE"
>> >
>> > finalize() {
>> > pgrep -fl "cat fifo $FIFO_FILE" > /dev/stderr
>> > pgrep -fl sha256sum > /dev/stderr
>> >
>> > cat_pid="$(cat "$CAT_PID_FILE")"
>> > rm "$CAT_PID_FILE"
>> > kill -s SIGTERM "$cat_pid" && wait "$cat_pid"
>> > exit 0
>> > }
>> >
>> > trap finalize SIGTERM
>> > trap finalize SIGHUP
>> > trap finalize SIGINT
>> > trap finalize SIGQUIT
>> >
>> > # Start the cat process and capture its PID
>> > while [ -f "$CAT_PID_FILE" ]; do
>> > cat "$FIFO_FILE" & cat_pid=$!
>> > echo "$cat_pid" > "$CAT_PID_FILE"
>> > wait "$cat_pid"
>> > sleep 0.01
>> > done | sha256sum > /dev/stderr
>> >
>> > Here’s how I’m running it:
>> >
>> > docker run -it -v $PWD:/app alpine sh -xc '
>> > apk add bash procps
>> > export FIFO_FILE="$(mktemp -u -p /dev/shm fifo_$$.XXXXXX)"
>> > bash -x /app/checksum.sh &
>> > sleep 0.2
>> > jobs
>> > ps -ef --forest
>> > echo "First data to hash" > $FIFO_FILE
>> > sleep 0.2
>> > ps -ef --forest
>> > echo "More data to hash" > $FIFO_FILE
>> > kill %1
>> > sleep 1
>> > ‘
>> >
>> > Am I missing something obvious? If it’s possible without the loop that’d be great too but I found that if I connect sha256sum directly to the fifo it exits after the first write to the fifo.
>> >
>> > Should I learn how to use socat for this? From a few of the things I’ve read it sounds like it could make this simpler?
>>
>>
>>
>> > _______________________________________________
>> > kwlug-disc mailing list
>> > To unsubscribe, send an email to kwlug-disc-leave at kwlug.org <mailto:kwlug-disc-leave at kwlug.org>
>> > with the subject "unsubscribe", or email
>> > kwlug-disc-owner at kwlug.org <mailto:kwlug-disc-owner at kwlug.org> to contact a human being.
>>
>>
>> _______________________________________________
>> kwlug-disc mailing list
>> To unsubscribe, send an email to kwlug-disc-leave at kwlug.org <mailto:kwlug-disc-leave at kwlug.org>
>> with the subject "unsubscribe", or email
>> kwlug-disc-owner at kwlug.org <mailto:kwlug-disc-owner at kwlug.org> to contact a human being.
>>
> _______________________________________________
> kwlug-disc mailing list
> To unsubscribe, send an email to kwlug-disc-leave at kwlug.org <mailto:kwlug-disc-leave at kwlug.org>
> with the subject "unsubscribe", or email
> kwlug-disc-owner at kwlug.org <mailto:kwlug-disc-owner at kwlug.org> to contact a human being.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kwlug.org/pipermail/kwlug-disc_kwlug.org/attachments/20240728/09be1d86/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 1355 bytes
Desc: not available
URL: <http://mail.kwlug.org/pipermail/kwlug-disc_kwlug.org/attachments/20240728/09be1d86/attachment-0001.p7s>
More information about the kwlug-disc
mailing list