[GSoC] migration

[Bradley Giesbrecht]

> On Jul 19, 2017, at 3:43 PM, Umesh Singla <umeshksingla at macports.org> wrote:
> 
> Hi Brad,
> 
> With another week over, I have the following updates to share. 
> 
> 
> GSoC Week #7 (13 Jul - 19 Jul)
> 
> I moved the things to migrate action instead of 'restore' since 'restore' looks a subset of 'migrate', so can be dealt easily once migrate is over.

I’m not sure I understand and if I do understand I think I disagree.

Snapshot is a state of installed and active ports with their variants.
Restoring a snapshot would turn off all parts and install/activate the snapshot ports with variants.
Migration would be taking a new snapshot, uninstalling all ports, upgrading the port command for the new arch and finally restoring the last snapshot which would entail installing all ports from snapshot since we installed earlier.

Snapshot and restore should be able to be executed without migrate.

Reading below I think I misunderstand so please clarify.

> Agenda for last week:
> 
> The migrate action consisted of three parts, creating a new snapshot, uninstall installed ports, re-installing them for the last snapshot.
> 
> 1. Creating a new snapshot was over before, though some refactoring is still left.
> 
> 2. Regarding uninstalling installed ports, I took the help of reclaim.tcl which works with the migrate action now. It uses sort_portlist_by_dependendents procedure. So, all in all, it uses "registry_uninstall::uninstall".
> 
> 3. Re-installing the ports from the portlist. I took the help of restore_ports.tcl here, which works when called migrate. It uses sort_ports [1] function. 
> 
> 
> 
> When we install, is it fine to use [mportexec $workername $install_target] from migrate.tcl?

I’m sorry, I don’t know base well enough to answer this. Can you please ask Clemens or Josh?

> What took time is that these two functions at present take the port list in different formats. So, I tried debugging by simply printing on console method to know the formats of the two. 
> 
> I want to use the sorting function for dependencies only once instead of calling it twice while uninstalling and then re-installing. I'm still working on it and actually, close to solve this but thought, better send the weekly update first.

It is indeed better to send updates more frequently. I’d check with Clemens on this. He mentored a previous GSoC project which used libsolve for dependency calculations. Perhaps there is something to borrow or learn from that project?

> For the next one week:
> 
> 	• make appropriate changes to procedures according to one format.
> 	• write for “SELECT PORTS FROM SNAPSHOT ..”, i.e. connect it with the new tables we have.
> 	• try finishing the migrate action asap?

Sounds good.

> Points:
> 
> Since migrate action creates a snapshot within and then uses it to redo the whole state, I feel creating a snapshot here explicitly is redundant. 

Not sure I follow. I imaging migrate action would start by calling snapshot, register the snapshot id, uninstall all ports, reinstall the port command and finally call restore action with the snapshot id for the last snapshot.

> Internally snapshot also uses the same registry C functions to get the information first and then save to database as [registry::entry imaged] does, so instead of first retrieving from [registry::entry imaged], saving it in sqlite and then again fetching all the info using the snapshot-id we just got, can't we directly use [registry::entry imaged] here? Am I missing something?

Oh, maybe in this case we already have the list in memory? But when we upgrade the port command wouldn’t we then call the new port command and perform the restore with the upgrade port command?

> Regarding "selected" snapshot for 'restore', it looks fine.
> 
> New points which I made during pondering to save for the end:
> 
> 	• Add ui_debug statements
> 	• Catch some error-prone areas.
> 	• Remove the useless/structure comments and add actual doc-strings.
> 	• Add copyright notices
> 	• Using ui_msg instead of puts and else. Should probably add that.

Yes to all the above.

> Also, here's my view of restore_ports.tcl of what it does:
> 
> 	• macports prefix configs
> 	• umask 022
> 	• set portList [read_portlist $filename] - reads the portlist from file supplied or stdin
> 	• set operationList [sort_ports $portList]
> 		• sort_ports $portList
> 			• extracts name, version, variants, active for a port
> 			• sets it 1 in port_in_list if not there, else increments the count for that port
> 			• if port dependencies for the combination (port_name, its variants) doesn’t exist already, fetches them using dependenciesForPort(port_name, its variants), kind of recursively.
> 		• dependenciesForPort $port_name $variants
> 	• install_ports $operationList
> 		• checks $active for a port and use it to decide install_target
> 		• uses mportexec for installing
> 
> 
> Please find the work log here [2]. It is largely in work-in-progress state.
> 
> 
> 
> [1]: https://github.com/macports/macports-contrib/blob/master/restore_ports/restore_ports.tcl#L52
> 
> [2]: https://github.com/macports/macports-base/commits/gsoc17-migrate
> 
> 
> 
> Best Regards,
> 
> Umesh Singla

Very good letter Umesh, thanks for the update. I hope my replies and questions are useful.

Regards,
Bradley Giesbrecht (pixilla)