portlist.tcl: illegal byte sequence

[Jim DeLaHunt]

Jason:

I have been watching this thread with interest.

Given that "my corrupted file can't be read by anything, it seems", it 
occurs to me that may be the problem is in the filesystem's 
representation of the file structure, rather than in the bytes of the file.

Or, since you mention that this file is hosted on a ZFS filesystem, and 
I am only aware of ZFS filesystems in file servers rather than macOS 
systems, maybe the problem is in the file-serving software running on 
the server or a client on your mac. If the file is on a separate file 
server, one thing to try is to get to a shell on that computer, and 
examine the underlying file. Can tools on the server read the file's 
contents?  What are the contents there?

I had baffling problems with apparent file corruption of a file written 
to a ZFS Fileserver on a TrueNAS Core OS (based on BSD) connected by 
SMB. The problem turned out to be that the file name on the server file 
system was so long that the server's or client's SMB code couldn't 
handle it. For some reason, other software on my mac had been able to 
write the file, with a long name, to that same filesystem. Weird. My fix 
was to use the server's shell to shorten the filename. Then all was well 
on the mac side.  I don't claim that this is your problem, just that 
there are a class of baffling problems that have symptoms of corrupt 
file contents but are actually caused by file system misbehaviour.

I hope this gives you a helpful lead.  Best regards,
      —Jim DeLaHunt


On 2025-12-25 14:44, Jason Liu wrote:
> Here's what happens:
>
> catalina-vm% cmp -lb portlist.tcl 
> /opt/local/libexec/macports/lib/portlist1.0/portlist.tcl
> cmp: /opt/local/libexec/macports/lib/portlist1.0/portlist.tcl: Illegal 
> byte sequence
>
> It looks like the file on my system is hosed to the point where it's 
> not even recognized as a text file anymore. I can't read the file 
> using vi either, vi shows an empty buffer with the message:
>
> "portlist.tcl" [readonly][READ ERRORS] 0L, 0B
>
> Interestingly, both the file downloaded from GitHub and the 
> corrupted one on my system are both exactly 10,041 bytes in size. 
> There's no way to checksum the files though, since my corrupted file 
> can't be read by anything, it seems:
>
> catalina-vm% openssl dgst -sha256 
> /opt/local/libexec/macports/lib/portlist1.0/portlist.tcl
> Read error in /opt/local/libexec/macports/lib/portlist1.0/portlist.tcl
> C09D841001000000:error:8000005C:system library:file_read:Illegal byte 
> sequence:crypto/bio/bss_file.c:148:calling fread()
> C09D841001000000:error:10080002:BIO routines:file_read:system 
> lib:crypto/bio/bss_file.c:150:
>
> -- 
> Jason Liu
>
>
> On Thu, Dec 25, 2025 at 3:34 PM Joshua Root <jmr at macports.org> wrote:
>
>     Diffing your portlist.tcl with a fresh copy downloaded from
>     <https://github.com/macports/macports-base/blob/v2.11.6/src/portlist1.0/portlist.tcl>
>
>     will show you exactly what has changed at least, which may give some
>     further clue.
>
>     - Josh
>
>     On 26/12/2025 03:22, Jason Liu wrote:
>     > That's a good point. In fact, the modification timestamp on
>     portlist.tcl
>     > is 2025-10-29 05:01, which would seem to indicate that the file
>     hasn't
>     > been touched for the past couple of months.
>     >
>     > The "illegal byte sequence" error does point to file corruption,
>     but the
>     > mystery is how that might have occurred. I did also check the
>     ZFS volume
>     > where the virtual disk for my MacPorts development VM is stored,
>     and
>     > none of the the weekly scrubs have repaired any bits, nor has any
>     > resilvering been performed, which are typically early indicators
>     that a
>     > drive might be failing.
>     >
>     > --
>     > Jason Liu
>     >
>     >
>     > On Wed, Dec 24, 2025 at 9:06 PM Ryan Carsten Schmidt
>     > <ryandesign at macports.org <mailto:ryandesign at macports.org>> wrote:
>     >
>     >     On Dec 24, 2025, at 18:49, Jason Liu wrote:
>     >>
>     >>     
>     >>
>     >>         portlist.tcl is a text file, not created by clang, and
>     Jason
>     >>         didn't mention running selfupdate so there's no reason why
>     >>         that file should have been changed.
>     >>
>     >>
>     >>     Sorry, I forgot to mention that. This did, in fact, occur
>     after a
>     >>     selfupdate. I have my MacPorts development VMs run a selfupdate
>     >>     every night around 5:00am-ish.
>     >
>     >     Ok but unless that selfupdate resulted in MacPorts base being
>     >     updated, that file wouldn't have been touched.
>     >
>     >     And if you run selfupdate daily then you'd have updated to the
>     >     latest version 2.11.6 weeks ago when it was released.
>     >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.macports.org/pipermail/macports-dev/attachments/20251225/7766f1d2/attachment.htm>