Understanding what I am calling the mess that is perl modules

[Scott Haneda]

On Jan 21, 2009, at 1:09 AM, Ryan Schmidt wrote:
> On Jan 21, 2009, at 02:35, Scott Haneda wrote:
>
>> I need Tie::RDBM
>> If someone can tell me anything wrong I am doing, or better ways, I  
>> would be able to provide more ports back to MacPorts.
>>
>> `port search p5-tie`
>> 4 results, none of which appear to be what I want.
>>
>> What are the possibilities Tie::RDBM is contained within one of the  
>> 4 results macports found?  The answer to that should be 0.  If it  
>> is not, I am searching wrong and would like to know how to  
>> correctly search.  If I am searching correctly, there needs to be a  
>> standard way to resolve this where the contents of a module are put  
>> into the port file so it can be searched and found.
>>
>> I have a feeling the answer is not 0.  That being the case, I would  
>> like to suggest no port be approved until this qualification for  
>> search is met.  ports may have 5000+ port files, but with this  
>> issue solved, it could appear to have many many more.
>
> I'm not sure this issue is limited to Perl modules. What if a user  
> knows that they want the program "mogrify", but they do not know  
> what software package provides it? "port search mogrify" does not  
> tell you. You have to already know that it's provided by  
> ImageMagick. MacPorts at this time seems to assume that the user  
> either already knows what software package they want to install, or  
> that they can find it using keywords from the description.
>
> It might be nice to let the user search by what files are in the  
> port. "port contents" tells you what's in a port, but only if it's  
> already installed, which isn't very helpful if you're trying to  
> figure out which port to install. Problem is the list of files in a  
> port is not known to MacPorts until the port is installed. This  
> could be more easily done if we had binary packages, but that is a  
> long way off. There are also the occasional ports that install  
> different files on different systems; one would have to take care  
> that all possible files are included in the search.

Wow, `port contents` is pretty damn handy.  I see your points.   
However, and maybe it is just me looking at this from the perspective  
of a user who does not know much about this, much like me...

How did I get here?
I want to make an ASSP port, so I downloaded it, I learned it is a  
pretty simple port.  In all reality, it may not even need a port, it  
works fine with Apple's perl, and it is nothing more than a copy of  
some files and a single command to get it up and running.  In this  
case, a port my be total overkill for ASSP.

But... ASSP needs about 15 perl modules to work, and maybe even 20 or  
so if you want anti-virus.  This led me here.  I wanted to make a port  
since my past experience with php and mysql here was pretty nice.  I  
thought I would give back and also to my own gain.

There is CPAN, and what is troublesome about all this, is the ASSP  
distro has a simple install file you can run, and it runs down the  
CPAN tree and gets what it needs.  I have always been wary of CPAN,  
since I am under the impression it hooks into the Apple perl, and for  
an email server proxy, the last thing I need is for Apple to break my  
kit.

I guess there is Fink, but somewhere I also got the impression  
MacPorts was sort of the new Fink.

ASSP lists the perl modules I need.  I am not sure why the developer  
lists them out as sub modules, if you can not get to them without  
getting the entire master module as well.  He seems to know what he is  
doing, so I would guess many other developers just list the one module  
they are using, rather than the entire package.

I, as a end user of MacPorts, got pretty hung up on this.  If I wanted  
to use ASSP, I would just download it, and if I wanted to hand install  
the modules it needs via CPAN, I just copy and paste the list out of  
the ASSP notes.

In MacPorts, I can not do that.  I have to google the port ASSP wants,  
look to see if it is in MacPorts, if not, google again for the main  
package the module may be contained in, then see if MacPorts knows  
about that.  If not, I can then think about making a port file.

How is CPAN solving this?  Even if there is no master list of files,  
so we could get to the result of `port contents` ahead of time, their  
site seems so structured it would should not be too hard to get to  
that list.  I just do not know enough about this, and I am not really  
looking for an answer from you, but more just to relay my experience  
so you may take it under consideration.

This will be a gross simplification, but take LWP::Simple, if I need  
that, I learn it is inside libwww-perl...

Gisle Aas > libwww-perl-5.823 > LWP::Simple

That is just a copy and paste of the breadcrumbs from their cpan  
site.  While it would be a mess, just backing up one on the breadcrumb  
gets you where you need to be.  I am not saying to html scrape the  
site, but you get what I mean. The process is a burden, there has to  
be a way to solve this for new users in order to attract more users.

Maybe, at the very least, add a new category, such as port_contains.   
When I as a port creator make a port, I could then run `port contents`  
and put it into that variable.  Or heck, automate it, and make a `port  
prepare` command, which will create a port file ready to submit to  
trac, or even just sent it in, over the wire.

>> I am trying to add in Tie::RDBM and find this page:
>> http://search.cpan.org/~lds/Tie-DBI-1.02/lib/Tie/RDBM.pm
>>
>> Version 0.70, so I would make my port file with a name of:
>> p5-Tie-RDBM 0.70
>> Which is going to translate to downloading a file of:
>> Tie-RDBM-0.70.tar.gz
>>
>> I am betting dollars to donuts that will fail.  On the cpan page I  
>> see they want me to download Tie-DBI-1.02.tar.gz.
>>
>> And there, I am lost, the version is way off, the name is off,  
>> there is no way MacPorts is going to figure this out.
>
> So the MacPorts port should probably be for the entire Tie::DBI 1.02  
> package, not just for its Tie::RDBM subpackage. You can ask the  
> author(s) of Tie::DBI why they have combined several perl modules  
> into one package, but presumably it's because they all work together  
> and require one another.

Understood.  Thanks.
I think part of this is, I have a tendency to want the bare minimum.   
I am getting over it, but others are much more strict.  The times I  
have watched someone use CPAN, to stop using it, just because all this  
stuff scrolls over the screen, and they walk away feeling there are  
now 100's to 1000's of files that were put into their system, and they  
will never get rid of them.

MacPorts solves this in that, at least, you know the files are all in  
one place, no matter how messy.  a drag of a folder more or less  
removes MacPorts sans a few dot files and maybe a startup item or  
launchd item.

My gut does not want me to get the entire parent package, I guess I  
just need to get over that.

>> I actually do not even understand why a port has to be made for  
>> anything in CPAN.  I am sure the publish a list of all their  
>> modules, how come that list can not just exist in MacPorts, and we  
>> can simply issue something like `sudo port install cpan- 
>> foo::bar::baz and have MacPorts do the rest.
>>
>> For items that would not build right away, or if someone wants  
>> something tried, true, and tested, there could be a full port file.
>
> I don't recall anyone having discussed such an idea before, so  
> that's probably one reason why it hasn't been done. We could have  
> that discussion now.
>
> One problem with this approach is that of the version number.  
> MacPorts ports are always of a particular version -- one which has  
> been tested by its maintainer, or at least, by someone. When a new  
> version is released, someone tests it and updates the port. The new  
> version may have new bugs the committer didn't see, the new version  
> may not even compile for everyone, but at least it was tested in  
> some way, presumably on a Mac, by somebody. If you just make  
> MacPorts a conduit to CPAN, there would be no testing of those  
> modules by a MacPorts user at all.

Ok, I understand that part.  And I hope my efforts are not being  
counterproductive, as what you describe above, all the bad parts, is  
what I am doing.  And I was almost proud of my work :)

I need a large handful of perl modules.  I do not generally code in  
perl, and I certainly have no intention of writing any test cases of  
the modules on my system.

I copy a stub portfile, edit the name of the folder to prepend p5- and  
replace :: with -.  From there, I will edit the port, change the name  
and version in the port, pop in my email address, copy and paste the  
two descriptions from the CPAN page, and drop in a url.

I then run `sudo port -d install` and let it error out, copy the  
checksums, put those in the port file, and install the bugger.

If it compiles, I call it a day, and submit it to trac.  Is this bad?   
I am not testing the module in any way, other than to make sure it  
compiles pretty clean.  And by "pretty clean" I just mean that as the  
compile is happening at a million lines a second on my screen, unless  
I catch something, I am just waiting to make sure it builds and I get  
a success message from the port install command.

Now, with ASSP, I will indeed test it and make it work, but these  
little bits of perl are just a means to an end for me.

Am I messing up a process with my methods?  It was not my intention to  
muddy up the port pool.

> Consider also how MacPorts works: there is a directory of portfiles,  
> and a portindex file that lists them all. "port install" consults  
> the index to know if the port exists. "port outdated" consults the  
> index to know if your ports are outdated. If there are no portfiles  
> for CPAN modules, how would they appear in the index? If they're not  
> in the index, how would "port outdated" work? Would we have a  
> separate index for CPAN modules?

I was thinking, if there is a portfile, it means a human made it, and  
took the time to make it work for the most part, even if only in the  
poor way in which I test it above.  So if there is a port file, that  
takes priority.  If not, fall back to automatic mode, to get the user  
the port file.  Grab the module, grab the most current version, build  
it out, and be done with it.  You would not consult the index in these  
cases, this would have to happen against a remote mapping database of  
sorts just for CPAN stuff.

> I'm sure there are several more issues to consider; those are just  
> off the top of my head. And of course again they're not limited to  
> Perl modules from CPAN. One could also consider how this could apply  
> to PHP modules from PEAR and PECL, Ruby gems, Octave packages, etc.


Indeed, and again, I do not know too much about this stuff.  I fall in  
the middle of all this, where I spend most of my day sysadmin, or  
developing in the actual code.

Thanks again for having this dialogue.  I am indeed making progress  
and learning about this system.  It was a long and windy road to get  
to a `sudo port install assp` but I am getting there.
--
Scott