read() on dir

[Peter West]

Thanks, Richard, for the history lesson.

peter
—
pbw at ehealth.id.au
“…an hour is coming when all who are in the tombs will hear his voice and come out…”

> On 18 Mar 2021, at 2:50 am, Richard L. Hamilton <rlhamil at smart.net> wrote:
> 
> I think you may be mixing apples and oranges.
> 
> As I recall, the ability to literally use read() began in very early Unix, when only one filesystem type existed (before SVR3 FSS or BSD VFS mechanisms that allowed multiple filesystem implementations). The original type had 16-byte entries, with two bytes of inode number (pretty small number of inodes/files that allows on a filesystem!) and fourteen bytes of file name, null byte padded on the right, but only null byte terminated if less than fourteen bytes. That was easy, and not too demanding on old, small systems. Note that mkdir() was not even a separate system call early on, it was just mknod() with the arguments needed to create an empty directory, followed by link() calls (allowed as root, mkdir command was setuid) to create "." and ".." links. rmdir simply checked that only those two links remained, unlinked them, and then unlinked the once-again empty (logically) directory.
> 
> But it had some disadvantages, like fragmentation. BSD UFS uses a data structure that resembles (but is not identical to) what readdir() returns.
> 
> Both of those (even once VFS came into existence) allowed actual read() calls on a directory for backwards compatibility, ONLY on certain filesystem types (SysV FS variants and UFS, mainly). Since different filesystems could implement directories differently (or even not really implement them at all, just implement the illusion of them in some indexing scheme supplemented with attributes for the directory nodes - what HFS, APFS, ZFS, etc do, more or less), and since sharing filesystems like NFS abstracted the directory structure anyway (hiding the "real" structure on the server), newer filesystems disallow direct read() of directories; it's really not to useful for anything other than maybe a low-level filesystem debugger or repair tool to know how directories are stored.
> 
> Along with FSS or VFS introducing readdir() (or actually getdents() or getdirentries(), on top of which readdir() was a library routine), mkdir() and rmdir() became system calls (no setuid helper needed anymore), read() of directories was as I said mostly disallowed except for compatibility on the oldest filesystem types, and link() or unlink() on directories mostly disallowed even for root (I think). With the backwards compatibility exceptions, that make directories abstract enough to impose no particular limits on how they were implemented, save that they had to handle a minimum file length, and have either per-directory attributes (permission bits, UID, GID, timestamps, etc) or at least have the filesystem fake those when they didn't really exist (as PCFS does).
> 
> That discussion roughly (I probably got some details wrong) covers the history of physically reading a directory. But what vi (vim, really) does is quite different. It simply stat()s a pathname, and if that reveals that the pathname is a directory, it does a readdir(), and stat()s each of the returned entries to discover if it's a file or directory, and then presents a listing upon which one can perform certain actions (scroll through it, hit return to edit/view the file or directory, etc). That is code within vim, and does NOT require or depend on being able to read() a directory. Other programs do similar things; for example, Finder and/or launch services on a Mac can "open" a fairly arbitrary pathname or even some URL types, but certainly does not depend on a single system call or sometimes even a single program to provide the behavior; it's a higher level abstraction, is all.
> 
> Other vi-like programs (original vi and its code base descendants such as the nvi port, elvis, vile, viper, etc) may or may not have a feature similar to the :Explorer or directory "editing" feature in vim.
> 
> It's quite a different question, a philosophical one if you will, whether a text editor should have a minimal file manager capability, as vim does. I don't see why not, if you don't mind a somewhat bigger code base and executable, since it already has :n and :N (or :prev) commands, allowing one to step through a list of files - not like it's all THAT much extra code required.
> 
> 
>> On Mar 17, 2021, at 11:19, Christoph Kukulies <kuku at kukulies.org <mailto:kuku at kukulies.org>> wrote:
>> 
>> There was some discussion in the FreeBSD mailing list about changing the behaviour of a directory fd WRT
>> the read() function. A change has been made towards disallowing this from FreeBSD 12.2 onwards (IIRC) and there was big discussion on this since it
>> „violates“ the cleanness of UNIX, that a file is a file is a file or „everything is a file“. Many purists were against the disallowing behaviour.
>> 
>> At that time I cross checked whether this is allowed in macOS and - again IIRC - at that time (Catalina) it was disallowed either to do
>> a „vi .“:
>> 
>> $ vi /tmp
>> 
>> " ============================================================================
>> " Netrw Directory Listing                                        (netrw v168)
>> "   /tmp
>> "   Sorted by      name
>> "   Sort sequence: [\/]$,\<core\%(\.\d\+\)\=\>,\.h$,\.c$,\.cpp$,\~\=\*$,*,\.o$,\.obj$,\.info$,\.swp$,\.bak$,\~$
>> "   Quick Help: <F1>:help  -:go up dir  D:delete  R:rename  s:sort-by  x:special
>> " ==============================================================================
>> ../                                                                                                                            
>> ./
>> .vbox-kuku-ipc/
>> com.apple.launchd.AUnEPqi6tC/
>> com.apple.launchd.atGF3BeR4X/
>> com.apple.launchd.lOdpt02m8q/
>> powerlog/
>> FirstBootAfterUpdate
>> FirstBootCleanupHandled
>> OSL_PIPE_501_SingleOfficeIPC_57c1e9acbaf815f47e314f3cbee8d6=
>> fseventsd-uuid
>> 
>> Now I’m surprised that this is possible (again?) under BigSur. I may be totally wrong on this. I don’t have the
>> opportunity to cross check that against Catalina. Could someone confirm or proove the opposite?
>> 
>> —
>> Christoph 
>> a UNIX dinosaur
>> 
>> 
>> 
>> 
>> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.macports.org/pipermail/macports-users/attachments/20210318/985d0703/attachment.htm>