salt-master: error: Failed to load configuration: Cannot load native module 'Cryptodome.Cipher._raw_des'

Romo

Getting this error on my salt-master (CentOS 7), which is causing it to not being able to start.

 salt-master: error: Failed to load configuration: Cannot load native module 'Cryptodome.Cipher._raw_des': Trying '_raw_des.so': /usr/lib64/python2.7/site-packages/Cryptodome/Util/../Cipher/_raw_des.so: undefined symbol: des_ecb_decrypt, Trying '_raw_desmodule.so': /usr/lib64/python2.7/site-packages/Cryptodome/Util/../Cipher/_raw_desmodule.so: cannot open shared object file: No such file or directory

Anybody has a clue what could be causing this?

Emad R

@romo said in salt-master: error: Failed to load configuration: Cannot load native module 'Cryptodome.Cipher._raw_des':

Getting this error on my salt-master (CentOS 7), which is causing it to not being able to start.

salt-master version ? and was it from repo of salt or packages of centos ?

I think getting the repo from salt is better

momurda

It is obviously waiting for opponent for a fight to the death.
"Two cipher enter; one cipher leave."

Obsolesce

Well, the last part of the error says cannot open shared object file: No such file or directory. So something relating to that I'd assume.

Romo

@emad-r said in salt-master: error: Failed to load configuration: Cannot load native module 'Cryptodome.Cipher._raw_des':

@romo said in salt-master: error: Failed to load configuration: Cannot load native module 'Cryptodome.Cipher._raw_des':

Getting this error on my salt-master (CentOS 7), which is causing it to not being able to start.

salt-master version ? and was it from repo of salt or packages of centos ?

I think getting the repo from salt is better

salt-master is 2018.3.1, used the bootstrap install so salt repo.

matteo nunziati

You seem to miss some encryption libs...

Romo

@matteo-nunziati said in salt-master: error: Failed to load configuration: Cannot load native module 'Cryptodome.Cipher._raw_des':

You seem to miss some encryption libs...

It does look like it, but why? Several months of everything working properly.

Tried forcing the reinstall of the salt and salt-master packages but still no luck.

Obsolesce

@romo said in salt-master: error: Failed to load configuration: Cannot load native module 'Cryptodome.Cipher._raw_des':

It does look like it, but why? Several months of everything working properly.

Any recent updates before you noticed it not working?

Heiko

I encountered the same problem today upgrading from salt-minion-2017.7.5-1 to salt-minion-2017.7.6-1 on an Redhat Enterprise Linux 7.
The minion won't start and doesn't log anything.
systemctl tells about:
salt-minion: error: Failed to load configuration: Cannot load native module 'Cryptodome.Cipher._raw_des': Trying '_raw_des.so': /usr/li...

I guess the problem is not staltstack itself but the Cryptodome lib that received an update aswell.

python -m Cryptodome.SelfTest
Traceback (most recent call last):
  File "/usr/lib64/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib64/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/usr/lib64/python2.7/site-packages/Cryptodome/SelfTest/__main__.py", line 30, in <module>
    SelfTest.run(stream=sys.stdout, verbosity=1, config= {'slow_tests' : slow_tests })
  File "/usr/lib64/python2.7/site-packages/Cryptodome/SelfTest/__init__.py", line 60, in run
    tests = get_tests(config=config)
  File "/usr/lib64/python2.7/site-packages/Cryptodome/SelfTest/__init__.py", line 81, in get_tests
    from Cryptodome.SelfTest import Cipher; tests += Cipher.get_tests(config=config)
  File "/usr/lib64/python2.7/site-packages/Cryptodome/SelfTest/Cipher/__init__.py", line 36, in get_tests
    from Cryptodome.SelfTest.Cipher import test_DES3;     tests += test_DES3.get_tests(config=config)
  File "/usr/lib64/python2.7/site-packages/Cryptodome/SelfTest/Cipher/test_DES3.py", line 30, in <module>
    from Cryptodome.Cipher import DES3
  File "/usr/lib64/python2.7/site-packages/Cryptodome/Cipher/DES3.py", line 57, in <module>
    """)
  File "/usr/lib64/python2.7/site-packages/Cryptodome/Util/_raw_api.py", line 191, in load_pycryptodome_raw_lib
    raise OSError("Cannot load native module '%s': %s" % (name, ", ".join(attempts)))
OSError: Cannot load native module 'Cryptodome.Cipher._raw_des3': Trying '_raw_des3.so': /usr/lib64/python2.7/site-packages/Cryptodome/Util/../Cipher/_raw_des3.so: undefined symbol: des3_ecb_decrypt, Trying '_raw_des3module.so': /usr/lib64/python2.7/site-packages/Cryptodome/Util/../Cipher/_raw_des3module.so: cannot open shared object file: No such file or directory

After downgrading to the previous version, salt-minion (2017.7.6-1) starts again.

yum downgrade python2-pycryptodomex-3.4.3-2.el7.x86_64

However downgrading that package is only a workaround and it will hopefully be fixed soon.

Romo

@heiko said in salt-master: error: Failed to load configuration: Cannot load native module 'Cryptodome.Cipher._raw_des':

I encountered the same problem today upgrading from salt-minion-2017.7.5-1 to salt-minion-2017.7.6-1 on an Redhat Enterprise Linux 7.
The minion won't start and doesn't log anything.
systemctl tells about:
salt-minion: error: Failed to load configuration: Cannot load native module 'Cryptodome.Cipher._raw_des': Trying '_raw_des.so': /usr/li...

I guess the problem is not staltstack itself but the Cryptodome lib that received an update aswell.

python -m Cryptodome.SelfTest
Traceback (most recent call last):
  File "/usr/lib64/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib64/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/usr/lib64/python2.7/site-packages/Cryptodome/SelfTest/__main__.py", line 30, in <module>
    SelfTest.run(stream=sys.stdout, verbosity=1, config= {'slow_tests' : slow_tests })
  File "/usr/lib64/python2.7/site-packages/Cryptodome/SelfTest/__init__.py", line 60, in run
    tests = get_tests(config=config)
  File "/usr/lib64/python2.7/site-packages/Cryptodome/SelfTest/__init__.py", line 81, in get_tests
    from Cryptodome.SelfTest import Cipher; tests += Cipher.get_tests(config=config)
  File "/usr/lib64/python2.7/site-packages/Cryptodome/SelfTest/Cipher/__init__.py", line 36, in get_tests
    from Cryptodome.SelfTest.Cipher import test_DES3;     tests += test_DES3.get_tests(config=config)
  File "/usr/lib64/python2.7/site-packages/Cryptodome/SelfTest/Cipher/test_DES3.py", line 30, in <module>
    from Cryptodome.Cipher import DES3
  File "/usr/lib64/python2.7/site-packages/Cryptodome/Cipher/DES3.py", line 57, in <module>
    """)
  File "/usr/lib64/python2.7/site-packages/Cryptodome/Util/_raw_api.py", line 191, in load_pycryptodome_raw_lib
    raise OSError("Cannot load native module '%s': %s" % (name, ", ".join(attempts)))
OSError: Cannot load native module 'Cryptodome.Cipher._raw_des3': Trying '_raw_des3.so': /usr/lib64/python2.7/site-packages/Cryptodome/Util/../Cipher/_raw_des3.so: undefined symbol: des3_ecb_decrypt, Trying '_raw_des3module.so': /usr/lib64/python2.7/site-packages/Cryptodome/Util/../Cipher/_raw_des3module.so: cannot open shared object file: No such file or directory

After downgrading to the previous version, salt-minion (2017.7.6-1) starts again.

yum downgrade python2-pycryptodomex-3.4.3-2.el7.x86_64

However downgrading that package is only a workaround and it will hopefully be fixed soon.

This is my current python2-pycryptodomex installed package, will try to downgrade as well to test it out.

Installed Packages
Name        : python2-pycryptodomex
Arch        : x86_64
Version     : 3.4.11
Release     : 2.el7
Size        : 16 M
Repo        : installed
From repo   : saltstack
Summary     : Self-contained Python package of low-level cryptographic
            : primitives
URL         : https://pycryptodome.readthedocs.io
License     : Public Domain and BSD
Description : PyCryptodome is a fork of PyCrypto. It brings the following
            : enhancements with respect to the last official version of PyCrypto
            : (2.6.1):
            : 
            : * Authenticated encryption modes (GCM, CCM, EAX, SIV, OCB)
            : * Accelerated AES on Intel platforms via AES-NI
            : * First class support for PyPy
            : * Elliptic curves cryptography (NIST P-256 curve only)
            : * Better and more compact API (nonce and iv attributes for
            :   ciphers, automatic generation of random nonces and IVs,
            :   simplified CTR cipher mode, and more)
            : * SHA-3 (including SHAKE XOFs) and BLAKE2 hash algorithms
            : * Salsa20 and ChaCha20 stream ciphers
            : * scrypt and HKDF
            : * Deterministic (EC)DSA
            : * Password-protected PKCS#8 key containers
            : * Shamir’s Secret Sharing scheme
            : * Random numbers get sourced directly from the OS (and not from a
            :   CSPRNG in userspace)
            : * Cleaner RSA and DSA key generation (largely based on FIPS 186-4)
            : * Major clean ups and simplification of the code base
            : 
            : PyCryptodome is not a wrapper to a separate C library like
            : OpenSSL. To the largest possible extent, algorithms are
            : implemented in pure Python. Only the pieces that are extremely
            : critical to performance (e.g. block ciphers) are implemented as C
            : extensions.
            : 
            : Python 2 version.

Romo

@romo said in salt-master: error: Failed to load configuration: Cannot load native module 'Cryptodome.Cipher._raw_des':

yum downgrade python2-pycryptodomex-3.4.3-2.el7.x86_64

Can't seem to downgrade

@salt ~]$ sudo yum downgrade python2-pycryptodomex-3.4.3-2.el7.x86_64
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: mirror.cenac.ipn.mx
 * epel: mirror.compevo.com
 * extras: repos-lax.psychz.net
 * updates: repos-tx.psychz.net
No package python2-pycryptodomex-3.4.3-2.el7.x86_64 available.
Error: Nothing to do

Romo

Downgraded the package using the full address to the previous version in the salt repo:

yum downgrade https://repo.saltstack.com/yum/redhat/7/x86_64/2016.11/python2-pycryptodomex-3.4.3-2.el7.x86_64.rpm

Fixes associated with Salt 2018.3.1 for Python 2 and Python 3 on Redhat 7 & 6 have been sent to QA.

scottalanmiller

Don't know about CentOS, but the fixed Salt packages hit the Fedora repos in the last few hours.