Unexpected skipped audits after hardening apache

[NickRycar]

Background:

I have been using the ssl-baseline profile to demonstrate a simple failure/remediation story using this cookbook to configure apache for ssl: https://github.com/chef-cft/bjc/tree/master/cookbooks/bjc-ecommerce

Recent updates look to produce an edge case where the profile produces an unexpected edge case, where the profile passes, but because it doesn't think any ports are listening on SSL, and skips all of the functional tests (e.g. there's one passed test for inspec version. The rest are skipped). Older versions of the profile do not appear to be affected.

Details & Reproduction Steps:

Initial revisions configured apache with the following settings to remediate detected errors:

SSLProtocol -all +TLSv1.2
SSLCipherSuite HIGH:!kRSA:!kDHr:!kDHd:!kSRP:!aNULL:!3DES:!MD5

Recent updates included a check for CBC ciphers (b5fd0ff), which the above settings do not alleviate. To address, we updated our CipherSuite to look like so:

SSLCipherSuite EECDH+AESGCM:EDH+AESGCM

Initial testing (done 4/17/17 - 4/18/17) seemed to validate that this fixed our remaining audits, and got a clean bill of health across the board. However, on Friday 4/21, we started seeing the above described behavior. Before enforcing the cipher suite, our tests fail as expected. After making the above change, tests now no longer detect that SSL is running, and all tests are skipped. Reverting back to an earlier version (on our case, from a 1.1.1 snapshot on our compliance server) produced the expected passing tests.

[supergicko]

hi @ChefRycar

which version of inspec do you use? maybe you have an old version, which does not include the fix of arlimus/sslshake#7 (underlying library for the inspec ssl resource)
Also related: arlimus/sslshake#6

[NickRycar]

Aha, that is a possibility. Do we know what version of inspec that fix was implemented in?

[supergicko]

@ChefRycar Yes

The updated sslshake library, which caused the problem was updated in inspec version >= 1.21.0

So with inspec 1.21.0 and 1.22.0 , this problem should go away

background:

Due to a bug in the underlying sslshake library arlimus/sslshake#6, arlimus/sslshake#7 - the ssl resource did not connect to servers configured with robust security (like enforced in this ssl-baseline) - resulting in skipping tests.

[NickRycar]

Thanks, @supergicko! That got me further along. Interestingly enough, I'm finding the opposite to be true -- when I'm using an earlier version of inspec, it detects the ports as expected, but with 1.22, it's skipping the ports. Here's a couple of tests I did in kitchen with differing inspec versions:

Inspec 1.11

Profile: DevSec SSL/TLS Baseline (ssl-benchmark)
Version: 1.2.0
Target:  ssh://[email protected]:22
  ✔  debugging: Inspec::Version=1.11.0
     ✔  tcpports=
     {:port=>22, :socket=>#<struct port=22, address="0.0.0.0", protocol="tcp", process="sshd", pid=1124, :listening?=nil>}
     {:port=>80, :socket=>#<struct port=80, address="::", protocol="tcp6", process="apache2", pid=14518, :listening?=nil>}
     {:port=>443, :socket=>#<struct port=443, address="::", protocol="tcp6", process="apache2", pid=14518, :listening?=nil>}
     {:port=>3306, :socket=>#<struct port=3306, address="127.0.0.1", protocol="tcp", process="mysqld", pid=12196, :listening?=nil>}
     {:port=>8009, :socket=>#<struct port=8009, address="::", protocol="tcp6", process="java", pid=14454, :listening?=nil>}
     {:port=>8080, :socket=>#<struct port=8080, address="::", protocol="tcp6", process="java", pid=14454, :listening?=nil>} should not eq nil
     ✔  sslports=
     {:port=>443, :socket=>#<struct port=443, address="::", protocol="tcp6", process="apache2", pid=14518, :listening?=nil>} should not eq nil
  ✔  ssl2: Disable SSL 2 from all exposed SSL ports.
     ✔  SSL/TLS on ec2-54-191-159-229.us-west-2.compute.amazonaws.com:443 with protocol == "ssl2" on node == ip-172-31-54-204 running "apache2" (14518)
     ✔  SSL/TLS on ec2-54-191-159-229.us-west-2.compute.amazonaws.com:443 with protocol == "ssl2" should not be enabled
  ✔  ssl3: Disable SSL 3 from all exposed SSL ports.
     ✔  SSL/TLS on ec2-54-191-159-229.us-west-2.compute.amazonaws.com:443 with protocol == "ssl3" on node == ip-172-31-54-204 running "apache2" (14518)
     ✔  SSL/TLS on ec2-54-191-159-229.us-west-2.compute.amazonaws.com:443 with protocol == "ssl3" should not be enabled
  ✔  tls1.0: Disable TLS 1.0 on exposed ports.
     ✔  SSL/TLS on ec2-54-191-159-229.us-west-2.compute.amazonaws.com:443 with protocol == "tls1.0" on node == ip-172-31-54-204 running "apache2" (14518)
     ✔  SSL/TLS on ec2-54-191-159-229.us-west-2.compute.amazonaws.com:443 with protocol == "tls1.0" should not be enabled
  ✔  tls1.1: Disable TLS 1.1 on exposed ports.
     ✔  SSL/TLS on ec2-54-191-159-229.us-west-2.compute.amazonaws.com:443 with protocol == "tls1.1" on node == ip-172-31-54-204 running "apache2" (14518)
     ✔  SSL/TLS on ec2-54-191-159-229.us-west-2.compute.amazonaws.com:443 with protocol == "tls1.1" should not be enabled
  ✔  tls1.2: Enable TLS 1.2 on exposed ports.
     ✔  SSL/TLS on ec2-54-191-159-229.us-west-2.compute.amazonaws.com:443 with protocol == "tls1.2" on node == ip-172-31-54-204 running "apache2" (14518)
     ✔  SSL/TLS on ec2-54-191-159-229.us-west-2.compute.amazonaws.com:443 with protocol == "tls1.2" should be enabled
  ✔  kx-ecdh: Enable ECDH as KX from all exposed SSL/TLS ports and versions.
     ✔  SSL/TLS on ec2-54-191-159-229.us-west-2.compute.amazonaws.com:443 with cipher == /^TLS_ECDH/i on node == ip-172-31-54-204 running "apache2" (14518)
     ✔  SSL/TLS on ec2-54-191-159-229.us-west-2.compute.amazonaws.com:443 with cipher == /^TLS_ECDH/i should be enabled
  ✔  kx-rsa: Disable RSA as KX from all exposed SSL/TLS ports and versions.
     ✔  SSL/TLS on ec2-54-191-159-229.us-west-2.compute.amazonaws.com:443 with cipher == /^TLS_RSA/i on node == ip-172-31-54-204 running "apache2" (14518)
     ✔  SSL/TLS on ec2-54-191-159-229.us-west-2.compute.amazonaws.com:443 with cipher == /^TLS_RSA/i should not be enabled
  ∅  kx-dh: Disable DH as KX from all exposed SSL/TLS ports and versions. (2 failed)
     ×  SSL/TLS on ec2-54-191-159-229.us-west-2.compute.amazonaws.com:443 with cipher == /^TLS_DH/i on node == ip-172-31-54-204 running "apache2" (14518)
     expected SSL/TLS on ec2-54-191-159-229.us-west-2.compute.amazonaws.com:443 with cipher == /^TLS_DH/i not to be enabled
     ×  SSL/TLS on ec2-54-191-159-229.us-west-2.compute.amazonaws.com:443 with cipher == /^TLS_DH/i should not be enabled
     expected SSL/TLS on ec2-54-191-159-229.us-west-2.compute.amazonaws.com:443 with cipher == /^TLS_DH/i not to be enabled
  ✔  kx-krb5: Disable KRB5 as KX from all exposed SSL/TLS ports and versions.
     ✔  SSL/TLS on ec2-54-191-159-229.us-west-2.compute.amazonaws.com:443 with cipher == /^TLS_KRB5/i on node == ip-172-31-54-204 running "apache2" (14518)
     ✔  SSL/TLS on ec2-54-191-159-229.us-west-2.compute.amazonaws.com:443 with cipher == /^TLS_KRB5/i should not be enabled
  ✔  kx-psk: Disable PSK as KX from all exposed SSL/TLS ports and versions.
     ✔  SSL/TLS on ec2-54-191-159-229.us-west-2.compute.amazonaws.com:443 with cipher == /^TLS_PSK/i on node == ip-172-31-54-204 running "apache2" (14518)
     ✔  SSL/TLS on ec2-54-191-159-229.us-west-2.compute.amazonaws.com:443 with cipher == /^TLS_PSK/i should not be enabled
  ✔  kx-gostr: Disable GOSTR as KX from all exposed SSL/TLS ports and versions.
     ✔  SSL/TLS on ec2-54-191-159-229.us-west-2.compute.amazonaws.com:443 with cipher == /^TLS_GOSTR/i on node == ip-172-31-54-204 running "apache2" (14518)
     ✔  SSL/TLS on ec2-54-191-159-229.us-west-2.compute.amazonaws.com:443 with cipher == /^TLS_GOSTR/i should not be enabled
  ✔  kx-srp: Disable SRP as KX from all exposed SSL/TLS ports and versions.
     ✔  SSL/TLS on ec2-54-191-159-229.us-west-2.compute.amazonaws.com:443 with cipher == /^TLS_SRP/i on node == ip-172-31-54-204 running "apache2" (14518)
     ✔  SSL/TLS on ec2-54-191-159-229.us-west-2.compute.amazonaws.com:443 with cipher == /^TLS_SRP/i should not be enabled
  ✔  au-ecdsa-rsa: Enable ECDSA or RSA as AU from all exposed SSL/TLS ports and versions.
     ✔  SSL/TLS on ec2-54-191-159-229.us-west-2.compute.amazonaws.com:443 with cipher == /(RSA|ECDSA)_WITH/i on node == ip-172-31-54-204 running "apache2" (14518)
     ✔  SSL/TLS on ec2-54-191-159-229.us-west-2.compute.amazonaws.com:443 with cipher == /(RSA|ECDSA)_WITH/i should be enabled
  ✔  au-anon: Disable ANON as AU from all exposed SSL/TLS ports and versions.
     ✔  SSL/TLS on ec2-54-191-159-229.us-west-2.compute.amazonaws.com:443 with cipher == /ANON_WITH/i on node == ip-172-31-54-204 running "apache2" (14518)
     ✔  SSL/TLS on ec2-54-191-159-229.us-west-2.compute.amazonaws.com:443 with cipher == /ANON_WITH/i should not be enabled
  ✔  au-dss: Disable DSS as AU from all exposed SSL/TLS ports and versions.
     ✔  SSL/TLS on ec2-54-191-159-229.us-west-2.compute.amazonaws.com:443 with cipher == /DSS_WITH/i on node == ip-172-31-54-204 running "apache2" (14518)
     ✔  SSL/TLS on ec2-54-191-159-229.us-west-2.compute.amazonaws.com:443 with cipher == /DSS_WITH/i should not be enabled
  ✔  au-psk: Disable PSK as AU from all exposed SSL/TLS ports and versions.
     ✔  SSL/TLS on ec2-54-191-159-229.us-west-2.compute.amazonaws.com:443 with cipher == /PSK_WITH/i on node == ip-172-31-54-204 running "apache2" (14518)
     ✔  SSL/TLS on ec2-54-191-159-229.us-west-2.compute.amazonaws.com:443 with cipher == /PSK_WITH/i should not be enabled
  ✔  au-export: Disable EXPORT as AU from all exposed SSL/TLS ports and versions.
     ✔  SSL/TLS on ec2-54-191-159-229.us-west-2.compute.amazonaws.com:443 with cipher == /((EXPORT)(\d*)_WITH)/i on node == ip-172-31-54-204 running "apache2" (14518)
     ✔  SSL/TLS on ec2-54-191-159-229.us-west-2.compute.amazonaws.com:443 with cipher == /((EXPORT)(\d*)_WITH)/i should not be enabled
  ✔  enc-aes-gcm-chacha20: Enable AES256 or AES128 or AES256-GCM or AES128-GCM or CHACHA20 as Enc
     ✔  SSL/TLS on ec2-54-191-159-229.us-west-2.compute.amazonaws.com:443 with cipher == /WITH_(AES_256|AES_128|CHACHA20|AES_256_GCM|AES_128_GCM)/i on node == ip-172-31-54-204 running "apache2" (14518)
     ✔  SSL/TLS on ec2-54-191-159-229.us-west-2.compute.amazonaws.com:443 with cipher == /WITH_(AES_256|AES_128|CHACHA20|AES_256_GCM|AES_128_GCM)/i should be enabled
  ✔  enc-cbc: Disable CBC as ENC from all exposed SSL/TLS ports and versions.
     ✔  SSL/TLS on ec2-54-191-159-229.us-west-2.compute.amazonaws.com:443 with cipher == /(WITH_(\w+)_(CBC))/i on node == ip-172-31-54-204 running "apache2" (14518)
     ✔  SSL/TLS on ec2-54-191-159-229.us-west-2.compute.amazonaws.com:443 with cipher == /(WITH_(\w+)_(CBC))/i should not be enabled
  ✔  enc-rc4: Disable RC4 as ENC from all exposed SSL/TLS ports and versions.
     ✔  SSL/TLS on ec2-54-191-159-229.us-west-2.compute.amazonaws.com:443 with cipher == /WITH_RC4/i on node == ip-172-31-54-204 running "apache2" (14518)
     ✔  SSL/TLS on ec2-54-191-159-229.us-west-2.compute.amazonaws.com:443 with cipher == /WITH_RC4/i should not be enabled
  ✔  enc-export: Disable EXPORT as ENC from all exposed SSL/TLS ports and versions.
     ✔  SSL/TLS on ec2-54-191-159-229.us-west-2.compute.amazonaws.com:443 with cipher == /WITH_EXPORT/i on node == ip-172-31-54-204 running "apache2" (14518)
     ✔  SSL/TLS on ec2-54-191-159-229.us-west-2.compute.amazonaws.com:443 with cipher == /WITH_EXPORT/i should not be enabled
  ✔  enc-des: Disable DES, 3DES as ENC from all exposed SSL/TLS ports and versions.
     ✔  SSL/TLS on ec2-54-191-159-229.us-west-2.compute.amazonaws.com:443 with cipher == /(WITH_(\d*)(des))/i on node == ip-172-31-54-204 running "apache2" (14518)
     ✔  SSL/TLS on ec2-54-191-159-229.us-west-2.compute.amazonaws.com:443 with cipher == /(WITH_(\d*)(des))/i should not be enabled
  ✔  enc-enull: Disable eNULL as ENC from all exposed SSL/TLS ports and versions.
     ✔  SSL/TLS on ec2-54-191-159-229.us-west-2.compute.amazonaws.com:443 with cipher == /WITH_NULL/i on node == ip-172-31-54-204 running "apache2" (14518)
     ✔  SSL/TLS on ec2-54-191-159-229.us-west-2.compute.amazonaws.com:443 with cipher == /WITH_NULL/i should not be enabled
  ✔  enc-camellia: Disable CAMELLIA as ENC from all exposed SSL/TLS ports and versions.
     ✔  SSL/TLS on ec2-54-191-159-229.us-west-2.compute.amazonaws.com:443 with cipher == /WITH_CAMELLIA/i on node == ip-172-31-54-204 running "apache2" (14518)
     ✔  SSL/TLS on ec2-54-191-159-229.us-west-2.compute.amazonaws.com:443 with cipher == /WITH_CAMELLIA/i should not be enabled
  ✔  enc-seed: Disable SEED as ENC from all exposed SSL/TLS ports and versions.
     ✔  SSL/TLS on ec2-54-191-159-229.us-west-2.compute.amazonaws.com:443 with cipher == /WITH_SEED/i on node == ip-172-31-54-204 running "apache2" (14518)
     ✔  SSL/TLS on ec2-54-191-159-229.us-west-2.compute.amazonaws.com:443 with cipher == /WITH_SEED/i should not be enabled
  ✔  enc-idea: Disable IDEA as ENC from all exposed SSL/TLS ports and versions.
     ✔  SSL/TLS on ec2-54-191-159-229.us-west-2.compute.amazonaws.com:443 with cipher == /WITH_IDEA/i on node == ip-172-31-54-204 running "apache2" (14518)
     ✔  SSL/TLS on ec2-54-191-159-229.us-west-2.compute.amazonaws.com:443 with cipher == /WITH_IDEA/i should not be enabled
  ✔  enc-aes-ccm: Disable AES-CCM from all exposed SSL/TLS ports and versions.
     ✔  SSL/TLS on ec2-54-191-159-229.us-west-2.compute.amazonaws.com:443 with cipher == /(WITH_AES_(\w+)_(CCM))/i on node == ip-172-31-54-204 running "apache2" (14518)
     ✔  SSL/TLS on ec2-54-191-159-229.us-west-2.compute.amazonaws.com:443 with cipher == /(WITH_AES_(\w+)_(CCM))/i should not be enabled
  ✔  mac-sha384-sha256-poly1305: Enable SHA384 or SHA256 or POLY1305 as Mac from all exposed SSL/TLS ports and versions.
     ✔  SSL/TLS on ec2-54-191-159-229.us-west-2.compute.amazonaws.com:443 with cipher == /_(SHA384|SHA256|POLY1305)$/i on node == ip-172-31-54-204 running "apache2" (14518)
     ✔  SSL/TLS on ec2-54-191-159-229.us-west-2.compute.amazonaws.com:443 with cipher == /_(SHA384|SHA256|POLY1305)$/i should be enabled
  ✔  mac-md5: Disable MD5 Mac from all exposed SSL/TLS ports and versions.
     ✔  SSL/TLS on ec2-54-191-159-229.us-west-2.compute.amazonaws.com:443 with cipher == /_MD5$/i on node == ip-172-31-54-204 running "apache2" (14518)
     ✔  SSL/TLS on ec2-54-191-159-229.us-west-2.compute.amazonaws.com:443 with cipher == /_MD5$/i should not be enabled
  ✔  mac-sha: Disable SHA(1) Mac from all exposed SSL/TLS ports and versions.
     ✔  SSL/TLS on ec2-54-191-159-229.us-west-2.compute.amazonaws.com:443 with cipher == /_SHA$/i on node == ip-172-31-54-204 running "apache2" (14518)
     ✔  SSL/TLS on ec2-54-191-159-229.us-west-2.compute.amazonaws.com:443 with cipher == /_SHA$/i should not be enabled
  ✔  mac-null: Disable NULL Mac from all exposed SSL/TLS ports and versions.
     ✔  SSL/TLS on ec2-54-191-159-229.us-west-2.compute.amazonaws.com:443 with cipher == /_NULL$/i on node == ip-172-31-54-204 running "apache2" (14518)
     ✔  SSL/TLS on ec2-54-191-159-229.us-west-2.compute.amazonaws.com:443 with cipher == /_NULL$/i should not be enabled
Profile Summary: 31 successful, 1 failures, 0 skipped
Test Summary: 62 successful, 2 failures, 0 skipped
>>>>>> ------Exception-------
>>>>>> Class: Kitchen::ActionFailed
>>>>>> Message: 1 actions failed.
>>>>>>     Verify failed on instance <default-ubuntu-1404>.  Please see .kitchen/logs/default-ubuntu-1404.log for more details
>>>>>> ----------------------
>>>>>> Please see .kitchen/logs/kitchen.log for more details
>>>>>> Also try running `kitchen diagnose --all` for configuration

Inspec 1.22

Profile: DevSec SSL/TLS Baseline (ssl-benchmark)                                                                                
Version: 1.2.0                                                                                                                  
Target:  ssh://[email protected]:22                                                      
                                                                                                                                
  [PASS]  debugging: Inspec::Version=1.22.0                                                                                     
     [PASS]  tcpports=                                                                                                          
     {:port=>22, :socket=>#<struct port=22, address="0.0.0.0", protocol="tcp", process="sshd", pid=1124, :listening?=nil>}      
     {:port=>80, :socket=>#<struct port=80, address="::", protocol="tcp6", process="apache2", pid=14519, :listening?=nil>}      
     {:port=>443, :socket=>#<struct port=443, address="::", protocol="tcp6", process="apache2", pid=14519, :listening?=nil>}    
     {:port=>3306, :socket=>#<struct port=3306, address="127.0.0.1", protocol="tcp", process="mysqld", pid=12196, :listening?=ni
     {:port=>8009, :socket=>#<struct port=8009, address="::", protocol="tcp6", process="java", pid=14455, :listening?=nil>}     
     {:port=>8080, :socket=>#<struct port=8080, address="::", protocol="tcp6", process="java", pid=14455, :listening?=nil>} shou
ot eq nil                                                                                                                       
     [PASS]  sslports=                                                                                                          
      should not eq nil                                                                                                         
  [SKIP]  ssl2: Disable SSL 2 from all exposed SSL ports.                                                                       
     [SKIP]  Skipped control due to only_if condition.                                                                          
  [SKIP]  ssl3: Disable SSL 3 from all exposed SSL ports.                                                                       
     [SKIP]  Skipped control due to only_if condition.                                                                          
  [SKIP]  tls1.0: Disable TLS 1.0 on exposed ports.                                                                             
     [SKIP]  Skipped control due to only_if condition.                                                                          
  [SKIP]  tls1.1: Disable TLS 1.1 on exposed ports.                                                                             
     [SKIP]  Skipped control due to only_if condition.                                                                          
  [SKIP]  tls1.2: Enable TLS 1.2 on exposed ports.                                                                              
     [SKIP]  Skipped control due to only_if condition.                                                                          
  [SKIP]  kx-ecdh: Enable ECDH as KX from all exposed SSL/TLS ports and versions.                                               
     [SKIP]  Skipped control due to only_if condition.                                                                          
  [SKIP]  kx-rsa: Disable RSA as KX from all exposed SSL/TLS ports and versions.                                                
     [SKIP]  Skipped control due to only_if condition.                                                                          
  [SKIP]  kx-dh: Disable DH as KX from all exposed SSL/TLS ports and versions.                                                  
     [SKIP]  Skipped control due to only_if condition.                                                                          
  [SKIP]  kx-krb5: Disable KRB5 as KX from all exposed SSL/TLS ports and versions.                                              
     [SKIP]  Skipped control due to only_if condition.                                                                          
  [SKIP]  kx-psk: Disable PSK as KX from all exposed SSL/TLS ports and versions.                                                
     [SKIP]  Skipped control due to only_if condition.                                                                          
  [SKIP]  kx-gostr: Disable GOSTR as KX from all exposed SSL/TLS ports and versions.                                            
     [SKIP]  Skipped control due to only_if condition.                                                                          
  [SKIP]  kx-srp: Disable SRP as KX from all exposed SSL/TLS ports and versions.                                                
     [SKIP]  Skipped control due to only_if condition.                                                                          
  [SKIP]  au-ecdsa-rsa: Enable ECDSA or RSA as AU from all exposed SSL/TLS ports and versions.                                  
     [SKIP]  Skipped control due to only_if condition.                                                                          
  [SKIP]  au-anon: Disable ANON as AU from all exposed SSL/TLS ports and versions.                                              
     [SKIP]  Skipped control due to only_if condition.                                                                          
  [SKIP]  au-dss: Disable DSS as AU from all exposed SSL/TLS ports and versions.                                                
     [SKIP]  Skipped control due to only_if condition.                                                                          
  [SKIP]  au-psk: Disable PSK as AU from all exposed SSL/TLS ports and versions.                                                
     [SKIP]  Skipped control due to only_if condition.                                                                          
  [SKIP]  au-export: Disable EXPORT as AU from all exposed SSL/TLS ports and versions.                                          
     [SKIP]  Skipped control due to only_if condition.                                                                          
  [SKIP]  enc-aes-gcm-chacha20: Enable AES256 or AES128 or AES256-GCM or AES128-GCM or CHACHA20 as Enc                          
     [SKIP]  Skipped control due to only_if condition.                                                                          
  [SKIP]  enc-cbc: Disable CBC as ENC from all exposed SSL/TLS ports and versions.                                              
     [SKIP]  Skipped control due to only_if condition.                                                                          
  [SKIP]  enc-rc4: Disable RC4 as ENC from all exposed SSL/TLS ports and versions.                                              
     [SKIP]  Skipped control due to only_if condition.                                                                          
  [SKIP]  enc-export: Disable EXPORT as ENC from all exposed SSL/TLS ports and versions.                                        
     [SKIP]  Skipped control due to only_if condition.                                                                          
  [SKIP]  enc-des: Disable DES, 3DES as ENC from all exposed SSL/TLS ports and versions.                                        
     [SKIP]  Skipped control due to only_if condition.                                                                          
  [SKIP]  enc-enull: Disable eNULL as ENC from all exposed SSL/TLS ports and versions.                                          
     [SKIP]  Skipped control due to only_if condition.                                                                          
  [SKIP]  enc-camellia: Disable CAMELLIA as ENC from all exposed SSL/TLS ports and versions.                                    
     [SKIP]  Skipped control due to only_if condition.                                                                          
  [SKIP]  enc-seed: Disable SEED as ENC from all exposed SSL/TLS ports and versions.                                            
     [SKIP]  Skipped control due to only_if condition.                                                                          
  [SKIP]  enc-idea: Disable IDEA as ENC from all exposed SSL/TLS ports and versions.                                            
     [SKIP]  Skipped control due to only_if condition.                                                                          
  [SKIP]  enc-aes-ccm: Disable AES-CCM from all exposed SSL/TLS ports and versions.                                             
     [SKIP]  Skipped control due to only_if condition.                                                                          
  [SKIP]  mac-sha384-sha256-poly1305: Enable SHA384 or SHA256 or POLY1305 as Mac from all exposed SSL/TLS ports and versions.   
     [SKIP]  Skipped control due to only_if condition.                                                                          
  [SKIP]  mac-md5: Disable MD5 Mac from all exposed SSL/TLS ports and versions.                                                 
     [SKIP]  Skipped control due to only_if condition.                                                                          
  [SKIP]  mac-sha: Disable SHA(1) Mac from all exposed SSL/TLS ports and versions.                                              
     [SKIP]  Skipped control due to only_if condition.                                                                          
  [SKIP]  mac-null: Disable NULL Mac from all exposed SSL/TLS ports and versions.                                               
     [SKIP]  Skipped control due to only_if condition.                                                                          
                                                                                                                                
Profile Summary: 1 successful, 0 failures, 31 skipped                                                                           
Test Summary: 2 successful, 0 failures, 31 skipped                                                                              
       Finished verifying <default-ubuntu-1404> (0m7.06s).                                                                      
-----> Destroying <default-ubuntu-1404>...                                                                                      
       EC2 instance <i-02fc57d2e529a0cb7> destroyed.                                                                            
       Finished destroying <default-ubuntu-1404> (0m0.20s).                                                                     
       Finished testing <default-ubuntu-1404> (3m47.21s).                                                                       
-----> Kitchen is finished. (3m55.48s)                            

[supergicko]

@ChefRycar
i´ll spin up a kitchen instance with your CipherSuites

SSLCipherSuite EECDH+AESGCM:EDH+AESGCM right?

[NickRycar]

Yup!

[NickRycar]

Revisiting this, as I think this may be an issue upstream in inspec. Previously had pinned my environment to 1.19.x, which solved things in the short-term, but in updating my automate server to use the compliance beta, I needed to upgrade inspec to 1.27.x to get my results showing properly, and this issue re-surfaced.

As far as I can tell, the ssl resource isn't able to properly inspect SSL ports. To try and simply things, I pared down my usecase to a protocols-only set of resources, and scanned port 443 directly rather than dynamically portfinding. I found that inspec didn't seem to be able to correctly grab any of the protocol/cipher settings after updating my config. Anything I'm expecting to not be enabled passes, and anything I expect to be enabled fails.

Example:

As in the earlier case, I created a simplified ssl config in apache (e.g. I skipped the cipher settings, and focused on protocols alone), that looked like this:

SSLProtocol -all +TLSv1.2

Then, tried running a compliance profile that contained the following control:

control 'tls1.2' do
  title 'Enable TLS 1.2 on exposed ports.'
  impact 0.5
  describe ssl(host: 'localhost', port: 443).protocols('tls1.2') do
    it { should be_enabled }
  end
end

The control failed when I ran. If, however, I changed resources to invoke openssl directly on the node, it provided the expected passing result:

control 'tls1.2' do
  title 'Enable TLS 1.2 on exposed ports.'
  impact 0.5
  describe command('echo "EOF" | openssl s_client -connect 127.0.0.1:443/cart -tls1_2') do
    its('stdout') { should match /Secure Renegotiation IS supported/ }
  end
end

Suspect this is related to when ssl-handshake was updated around 1.20, but am unfamiliar with how things are controled under the hood, and haven't investigated further just yet.

[chris-rock]

@arlimus @supergicko any ideas?

[supergicko]

@chris-rock, @ChefRycar

i cannot reproduce this with chefdk 1.4.3 but this is a fresh install and i did not switch back and fort inspec.

@chris-rock is it possible to install a inspec version (for example the newest one) on top of chefdk? And if yes, does it use the sslshake IN inspec definition, or the sslshake of the chefdk (which could be an older one)? because to me it seems like an old version of sslshake is used to determine the handshakes, which leads to confusing results..

This issue needs some tests for different http(s) servers, maybe with docker