Commit Graph

55 Commits

Author SHA1 Message Date
Oliver Kurth 8544b87e14 reposync: urls only option, delete option, norepopath and check conflicting options 2021-05-28 12:18:44 -07:00
Oliver Kurth 52e5ade2f6 reposync: basics, metadata, metadata/download-path and option handling 2021-05-27 13:39:36 -07:00
Oliver Kurth a9c9e8e0c9 move special metalink handling out of TDNFDownloadFile() to caller 2021-05-27 13:36:46 -07:00
Prashant Singh Chauhan a05de169e3 print updateinfo and other warnings even if in quiet mode
print output of updateinfo and some other errors and warning
even if quiet mode is set.
2021-02-18 10:35:19 +00:00
Oliver Kurth 4f22970195 ignore coverity toctou errors 2021-01-21 11:39:31 -08:00
Oliver Kurth 2b339a500c add coverity comment to supress 'deadcode' error 2021-01-21 11:08:00 -08:00
Oliver Kurth 854e8ca0ba fix --cacheonly, add error code 2021-01-05 11:10:58 -08:00
Shreenidhi Shedi bd7fcc6d32 print improvements
Added a utilty function to print only if quiet not set
Also print errors irrespective of quiet option

If a non-exisitng package is provided for install/update let's show the
name of the invalid package in error message

Signed-off-by: Shreenidhi Shedi <sshedi@vmware.com>
2020-12-18 13:21:08 +05:30
Keerthana K 228a8d5113
Merge pull request #201 from tapakund/optimise_makecache_path
optimise makecache path
2020-12-18 12:43:36 +05:30
Tapas Kundu 441b376550 optimise makecache path
- In case of using baseurl:
  When we do makecache first time, tdnf will
  download the repomd.xml and copy all the
  repodata.
  From second time, if we do "makecache" or
  "repolist --refresh" we will check the
  shasum of repomd.xml with the existing
  repomd.xml and if both are not same, will
  replace the repodata.
  If we switch from metalink to baseurl. In that case
  we will download he repomd.xml and compare with
  the existing one. If both are same, we will not
  replace and use the existing one.

- In case of metalink:
  If we were using baseurl and switched to metalink.
  We will download metalink file and then download
  repomd.xml and compare the shasum with existing
  repomd.xml. If both matches we will not replace
  the repomd.xml. We will just keep the metalink file
  and save the baseurl.
  From next time onwards, we will check the shasum
  for metalink file, if it matches we will not proceed
  to download the repomd.xml.

Signed-off-by: Tapas Kundu <tkundu@vmware.com>
2020-12-10 11:05:36 +00:00
Shreenidhi Shedi 6f7d3cce32 Further fixes based on coverty scan report
Signed-off-by: Shreenidhi Shedi <sshedi@vmware.com>
2020-12-08 13:05:30 +05:30
Tapas Kundu 80192af1fa Fix Coverity Issues
- Fix mem leaks
- Declare variable at right place.

Signed-off-by: Tapas Kundu <tkundu@vmware.com>
2020-10-27 20:00:56 +05:30
oliverkurth f2793f3085 install local and remote RPMs from command line 2020-10-21 22:22:39 -07:00
oliverkurth 70b4523b77 import multiple keys 2020-10-01 11:24:59 -07:00
oliverkurth fec075dd86 make repo gpgkeys field an array of strings 2020-10-01 11:24:59 -07:00
Oliver Kurth f419d79d79 Download gpg keys from remote locations (http, ftp, ..).
The key is first downloaded into the cache area, then used
like a local key. The cached key will be deleted after it
has ben added to the key ring.
2020-09-21 11:36:10 -07:00
Tapas Kundu d4307085c6 print error string with error code
Signed-off-by: Tapas Kundu <tkundu@vmware.com>
2020-09-16 13:53:33 +08:00
Tapas Kundu fadaad452f support sequential retry of lesser priority mirrors
Below are the modifications done with this PR:

1. Add sha512sum support for verifying the repomd file.
2. Cache the baseurl in case of metalink.
3. support sequential retry of lesser priority mirrors
4. Removed hardcoded support of https, http, ftp, ftps, file resource types.
5. With this change, curl will decide on the protocol supported/unsupported.
6. Minor code cleanup and some improvements.

Signed-off-by: Tapas Kundu <tkundu@vmware.com>
2020-09-16 13:52:51 +08:00
oliverkurth 706cd2b682 address sshedi's comments 2020-08-31 11:03:40 -07:00
Oliver Kurth da4d89bf2a - test for gpgcheck before reading rpm
- make TDNFGetGPGSignatureCheck accept NULL for ppszUrlGPGKey
2020-08-18 15:35:45 -07:00
Tapas Kundu 4d8b5849ca
Skip debug logs if quiet option is set (#142)
Skip logging on console if tdnf quiet is set

Signed-off-by: Tapas Kundu tkundu@vmware.com
2020-05-29 22:58:42 +05:30
Tapas Kundu e6c10f6f17
Rename metalink file (#140)
- Removed hardcoded name for metalink file. It will depend on user input file.
- Fix some issues related with repomd url which is coming from metalink file.
- Moved check for resource type to TDNFParseAndGetURLFromMetalink before selecting
  the url.

Signed-off-by: Tapas Kundu <tkundu@vmware.com>
2020-05-29 13:24:30 +05:30
Keerthana K 56be3619bc
Fix stale solv cache issue (#134)
This fix is to remove the old repomd , solv cache and lastrefresh
marker when there is a new repomd file downloaded. This fixes the issue
https://github.com/vmware/photon/issues/1008

Signed-off-by: Keerthana K <keerthanak@vmware.com>
2020-05-12 10:28:43 +05:30
Tapas Kundu da56817f97 Improvement for metalink support
-Added checks for files and resources.
-Added verifying checksum for the files to be downloaded using metalink file
-If verifying checksum fails, we will fail the discard the download.
-Added checks for resource type to:
  - http
  - https
  - ftp
  - ftps
  - rsync
-We will also check the digest in below priority:
  - sha256
  - sha1
  - md5
If sha256 is valid in metalink file, we will store sha256 to compare with the downloaded repomd.
In case sha256 is invalid, we will store sha1 and use it for validating the downloaded repomd file.
If both sha1 or sha256 is not valid, we will store md5.

In case we have only one digest present in metalink file and if thats invalid, we will not store any
digest and discard by throwing error, digest missing.

Signed-off-by: Tapas Kundu <tkundu@vmware.com>
2020-04-21 20:48:47 +08:00
Tapas Kundu 8b5495dbc6 Add metalink support
Added metalink support for repo files.

If metalink URL is present, we will get the URL with the highest priority.
Then we concat the metalink path to the url and download the metalink file.

Once metalink file is downloaded, we parse it and downlod the repomd.xml
file and then parse the xml to download repoMD parts.

Also, we set the BaseURL for the respective repo to the URL we got from
the metalink file for further package downloads.

Signed-off-by: Tapas Kundu <tkundu@vmware.com>
2020-04-09 23:07:19 +08:00
Priyesh Padmavilasom 9b83081e53 when gpgcheck is enabled and gpgkey is missing in repo file,
installs used to fail with generic error. this was because
gpgkey lookup would fail with a null or empty string and a
generic error was returned.
it is a better user experience when the specific error is
bubbled up so that corrective action can be taken.
2020-04-07 19:47:02 +00:00
Shreenidhi Shedi 390a829234 Handle already installed packages faster.
Signed-off-by: Shreenidhi Shedi <sshedi@vmware.com>
2020-03-02 22:24:51 +05:30
Priyesh Padmavilasom d6a0e74723 plugin headers and initial repogpgcheck plugin implementation
- plugin event map for guidance to event items
 - plugin events init, repo, repomd
 - tests for plugin conf
 - tests with a repomd sign and verify via plugin
2020-02-12 18:50:26 +00:00
Shreenidhi Shedi c4de6ae9db Minor fixes all over the place.
Changes include:
1. Fix build warnings like unused parameters, sign mismatch while comparing.
2. Using printf instead of fprintf(stdout, ...).
3. Added header guard in few .h files.
4. Added a rule for silent build in configure.ac
5. Removal of trailing spaces, my vim does it automatically
6. Fixed few wrong date entries in spec files.
7. Removed unused function parameters in few places.

Signed-off-by: Shreenidhi Shedi <sshedi@vmware.com>
2019-10-14 22:25:38 +05:30
Priyesh Padmavilasom cf89f79c35 fix file handle leak
Change-Id: I4526d00d5b84b7c3ff30f6774e662ffeee80cf14
2019-07-10 18:16:01 +00:00
Keerthana K 7f23f9c2f5 Add skipsignature and skipdigest options
This change adds 2 new options for install / update commands:
    skipsignature - skip verifying RPM signatures
    skipdigest - skip verifying RPM digest

Change-Id: I3543ed8e9027755ec7cb3ecb49212596ac76f5a9
2019-04-12 08:31:11 +00:00
Keerthana K e6012ec141 changes in tdnf makecache and --refresh commands
Previously tdnf makecache and --refresh always downloads repodata irrespective
of changes in repodata. This change uses shasum comaparison to detect if there
is any change in repodata and downloads repodata only if there is any
change in repodata.

Change-Id: Ia800bb54e69fec649aa10188fe981d66d2fc56a7
2019-02-28 07:09:08 +00:00
Keerthana K 1c3108b829 Fix to add createwhatprovides() after repos are initialized.
createwhatprovides() was moved to InitSack() as part of this commit
355b46fb6f. createwhatprovides()
needs to be called after parsing all the enabled repos to get the index
of dependencies. Without calling this creates a problem in dependency list.

Change-Id: Idad468a73b48ef8374322011ceb64b68fc45e935
2019-02-18 08:56:24 +00:00
Keerthana K 355b46fb6f Fix to address errors when no repos are enabled.
pool_createwhatprovides is moved to InitSack() from InitRepo().
When no repos are enabled createwhatprovides is not set due to which
commands fail.

Change-Id: I3fc15022a9e6bfbcaec88c62343c856dba873861
2019-02-13 06:14:03 +00:00
Keerthana K e124b3654e Support for Libsolv caching of metadata
This change includes the support of libsolv caching of metadata which is a .solv file.
When the cache is created, tdnf creates a metadata cache file (*.solv)
in cache directory (/var/cache/tdnf/REPO-NAME/solvcache/). This is consumed by tdnf in
subsequent tdnf operations till the refresh timer expires. Solv cache file improves the
time consumed by tdnf operation which results in quicker results.

Change-Id: I42894260b8e9a110188c38027fc480dd83bfdc10
2018-11-30 06:47:01 +00:00
Priyesh Padmavilasom 15abaf339c Fix issues with libsolv merge
Change-Id: I51e1b5c7abc21c120bc294bf365a40c1e17c7c95
2018-01-08 15:38:03 -08:00
xiaolin-vmware 811e61fed9 merge uselibsolv branch.
Change-Id: I3e7e66d257897ade8319c562b9c9228a7c53af3b
2017-05-15 17:32:45 -07:00
Priyesh Padmavilasom fd0114c086 apply quiet flags to repo refresh downloads
Change-Id: I218566d1bafb78d9f2dd955636fb30889d06c3ee
2017-05-13 22:02:16 -07:00
Priyesh Padmavilasom bb7ac75bd7 add cli lib
Change-Id: I52b14eafe9e7ca4e1f9171b1bd20222c7467ee4e
2017-02-27 14:13:28 -08:00
Priyesh Padmavilasom 0fc25bd01a fix empty dir and dir remove issues in repo refresh
Change-Id: I53d730fe5179d36a1399f643513975e556216a58
2016-12-09 01:29:39 +00:00
xiaolin-vmware f25379c61e Remove glib dependency from tdnf.
Change-Id: I51ed0e224c54c2a3eff57b8c54af7204db2ef26e
2016-11-29 15:18:26 -08:00
Priyesh Padmavilasom ff092063d6 use libsolv for md parse, rewire downloads directly thru libcurl
Change-Id: Ib08cb8434820ed03b7995137045d5a4173493e7f
2016-11-19 14:41:07 +00:00
Priyesh Padmavilasom bc7fb5e040 add new options to help.
Change-Id: Idccdf856b9c5fa1e7fd0e72ad0f7290522464a7f
2016-11-05 01:09:10 +00:00
Priyesh Padmavilasom 583ef2ceb1 engage metadata_expire
Change-Id: I87a52796f5639d758a60b4707769cc71a06ac362
2016-06-24 08:20:53 +00:00
Priyesh Padmavilasom 6634fc1312 Display error for gpgkey access. Fix GetRepoById naming 2015-10-22 16:15:33 -04:00
Priyesh Padmavilasom 169b321918 Ignore gpg key config when gpgcheck is turned off 2015-09-08 03:57:23 +00:00
Priyesh Padmavilasom 492b4d9bc0 Add proxy support 2015-06-09 19:30:09 -04:00
Priyesh Padmavilasom 9835aed77f fix memory leaks 2015-05-31 17:51:16 -04:00
Priyesh Padmavilasom ef0725b01c Update license info 2015-04-15 17:10:08 -04:00
Priyesh Padmavilasom f65ed563d7 fix reponame comparison when finding repo by name 2015-04-14 23:23:38 -04:00