OT: cat and tail

Scott Haneda talklists at newgeo.com
Fri Dec 18 20:02:18 PST 2009


Totally ot, but it may end up in software that I put in a port, so  
here goes:

An app makes a log file, I have no control over it other than it is  
there, the developer is non responsive to change.

I run a script that does this
tail -f -n0 /path/to/log.log >> /path/to/log/running_log
The problem is, the log.log grows, it will get chopped in half by the  
app at a point in size I define, I set mine pretty high.  I want to  
get those log lines in MySql, parsing 100MB files is not fun.
So I run the tail line, which grabs a copy of log in real time, as it  
comes in. I was doing this next:
# make a copy of the file and also clear it cd /path/to/ cat  
running_log > stale_log && echo "" > running_log
Problem here, is I toss the cat into the bg, and who knows how long  
that can take, then I clear the running_log, but cat could still be  
chewing on it.
If I remove tossing it in the background, that helps a little, I will  
cat the file into a new log, which will be just the new tail'd data,  
and then clear it so it can start all over.  I then have stale_log to  
process later.
However, there is a chance, also, that while the cat is going on, new  
log lines are also being added as well.  I have considered, echo  
"unique-marker" >> running_log and then finding a way to only copy the  
lines before "unique-marker", but that seems messy, and I bet there is  
a handy one liner that can pull this off.
Thanks for any pointers, if the unique-marker is the way to go,  
suggestions on how to do that would be appreciated as well.
-- 
Scott * If you contact me off list replace talklists@ with scott@ * 


More information about the macports-users mailing list