IPA wait_for_open_ports: localhost [8080, 8443] timeout 300
Today we were updating the server with yum update on our IPA server.
The yum update finished successfully, but the IPA was not starting after the upgrade. the ipactl restart was also throwing an error. The web interface came up after the server restart but it was throwing “unknown error” while trying to log in via the web interface.
Below is the error from /var/log/ipaupgrade.log
2017-09-13T08:38:11Z DEBUG stderr= 2017-09-13T08:38:11Z DEBUG wait_for_open_ports: localhost [8080, 8443] timeout 300 2017-09-13T08:43:11Z ERROR IPA server upgrade failed: Inspect /var/log/ipaupgrade.log and run command ipa-server-upgrade manually. 2017-09-13T08:43:11Z DEBUG File "/usr/lib/python2.7/site-packages/ipapython/admintool.py", line 172, in execute return_value = self.run() File "/usr/lib/python2.7/site-packages/ipaserver/install/ipa_server_upgrade.py", line 46, in run server.upgrade() File "/usr/lib/python2.7/site-packages/ipaserver/install/server/upgrade.py", line 1913, in upgrade upgrade_configuration() File "/usr/lib/python2.7/site-packages/ipaserver/install/server/upgrade.py", line 1646, in upgrade_configuration upgrade_pki(ca, fstore) File "/usr/lib64/python2.7/contextlib.py", line 24, in __exit__ self.gen.next() File "/usr/lib/python2.7/site-packages/ipaserver/install/installutils.py", line 1134, in stopped_service service_obj.start(instance_name) File "/usr/lib/python2.7/site-packages/ipaplatform/redhat/services.py", line 211, in start instance_name, capture_output=capture_output, wait=wait) File "/usr/lib/python2.7/site-packages/ipaplatform/base/services.py", line 300, in start self.wait_for_open_ports(self.service_instance(instance_name)) File "/usr/lib/python2.7/site-packages/ipaplatform/base/services.py", line 270, in wait_for_open_ports self.api.env.startup_timeout) File "/usr/lib/python2.7/site-packages/ipapython/ipautil.py", line 1227, in wait_for_open_ports raise socket.timeout("Timeout exceeded") 2017-09-13T08:43:11Z DEBUG The ipa-server-upgrade command failed, exception: timeout: Timeout exceeded 2017-09-13T08:43:11Z ERROR Timeout exceeded 2017-09-13T08:43:11Z ERROR The ipa-server-upgrade command failed. See /var/log/ipaupgrade.log for more information
As recommended on the error log, we tried running ipa-server-upgrade command but it was throwing the same error as above.
- We have verified that the telnet on localhost port 8080 and 8443 is working fine.
- Tried disabling SELinux and firewalld and rebooted the server, The issue was not solved.
As we checked in detail, it turned out that the client had disabled IPv6 on the kernel as a part of PCI DSS Benchmarking. But the /etc/hosts entry for ipv6 localhost was still on the hosts file.
the solution was to remove the above entry from /etc/hosts file and rerun the