GSoC 15 - Dependency Calculation using SAT

[Jackson Isaac]

On Thu, Mar 19, 2015 at 3:25 AM, Clemens Lang <cal at macports.org> wrote:
> Hi,
>
> ----- On 17 Mar, 2015, at 14:43, Jackson Isaac jacksonisaac2008 at gmail.com wrote:
>
>> As you know the application period has started. Can you please help me
>> out with some things regarding the proposal.
>
> Certainly, I'd be happy to :)
>
>
>> I have built the latest Macports (i.e 2.3.99 from source) and as you
>> said I can see the confirmation before installing dependencies and
>> also it looks much cleaner now.
>>
>> Also I went through the source code. The dependency is build in
>> src/macports1.0/macports.tcl src/macports1.0/portdepends.tcl and
>> src/macports1.0/macports_dlist.tcl if I am not wrong or missing out
>> anything else.
>
> Nope, that's correct. The code is old, but works fine.
>
>
>> I have also gone through the CUDF at
>> http://www.mancoosi.org/cudf/primer/ and also
>> http://docs.codehaus.org/display/MAVEN/SAT+Based+Dependency+Resolution.
>>
>> Does using CUDF mean that our portfile format will have to be modified
>> (there is not much syntax difference, we might be able to interpret
>> the old portfiles to a cudf file format while running dependency
>> check.
>
> I'm not particularly attached to actually using CUDF. I sounded like a
> good idea when I last checked it, but it largely depends on the SAT
> solver we'd use in the end. I'm not sure the current CUDF-solvers have
> good support for specifying your own objective function to optimize.
>
> But anyway, regardless of the format we actually use for the SAT solver,
> a problem that might come up is actually converting Portfiles into a
> representation that can be used. As you correctly stated, the results
> of both formats are equivalent enough so that the actual conversion
> isn't hard. However, to generate accurate dependency information, we'd
> usually have to execute the Portfiles, and that can really slow the
> whole process down quite a bit, which we should avoid. From what I've
> seen, dependency resolution methods based on SAT solving expect the
> complete universe of available packages and their dependency relations
> plus a list of packages that should be installed after the current
> action finished. Generating the complete list with accurate dependency
> info by executing the Portfiles would take ~15 minutes and is thus
> out of the question.
>
> We do have a cache of the information, which we call the port index.
> However, since Portfiles can execute arbitrary code, some Portfiles
> may currently depend on the behavior that they are being evaluated
> during dependency calculation (e.g. by changing their depends_lib
> depending on the variant selection the user has given). We should
> figure out a solution for that, for example by
>  (a) somehow adding the effect of a variant on dependency relations
>      to the portindex, so we could tell what the dependencies of
>      foobar +bar +baz were without actually running the Portfile.

By variant do you mean the packages that are required by the package
that we want to install.

Do you mean like in Portfile we have:

packagename somenumber (I don't know what this number means) and
followed by description.

So here itself we add/cache the dependencies when we fetch the
portfiles (or maybe when we are trying to install to keep reference
for future).

So our new Portfile will look something like:

packageToBeInstalled somenumber +packageDepends someothernumberOfThatPackage
followed by description of the packages.

>  (b) Pre-generating a universe description file (e.g. in CUDF
>      format) and only updating it with accurate dependency
>      information of the port we're currently installing and its
>      dependencies. That would however, defeat the point of the
>      whole exercise, because we'd still have to figure out the
>      complete dependency tree before using a SAT solver to, well,
>      figure out the dependency tree.

So CUDF is out of the picture, as it seems to take more time to
compute than our obejective.

>  (c) Pre-generate the universe description and find a different
>      solution to deal with variants. There are a couple of
>      approaches that might work here, such as doing fixpoint
>      iteration of (1) figuring out dependencies using SAT solving,
>      (2) updating the dependency information of these ports taking
>      variants into account, (3) repeat until stable.
>

SAT Solving (as mentioned here:
http://docs.codehaus.org/display/MAVEN/SAT+Based+Dependency+Resolution)
or do you have something else too in mind.