Host DOWN alert for vmhost-x86-copr01.rdu-cc.fedoraproject.org!

Discussion:

Pavel Raiskup

2021-04-26 20:22:17 UTC

This is (probably) my fault. I played a bit with ipv6 && libvirt, and I
rebooted the machine, and it doesn't boot up now :-( sorry for the noise.

If there's some console (VPN?) access that we could use to fix such problems
within our (CPT) team?

Pavel

***** Nagios *****
Notification Type: PROBLEM
Host: vmhost-x86-copr01.rdu-cc.fedoraproject.org
State: DOWN
Address: vmhost-x86-copr01.rdu-cc.fedoraproject.org
Info: CHECK_NRPE STATE CRITICAL: Socket timeout after 30 seconds.
Source: noc01.iad2.fedoraproject.org
Date/Time: Mon Apr 26 20:06:41 GMT 2021

_______________________________________________
infrastructure mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-***@lists.fedoraproject.org
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/***@lists.fedoraproject.org
Do not reply to spam on the list, report it: https://pagure.i

Stephen John Smoogen

2021-04-26 20:44:29 UTC

Permalink

Post by Pavel Raiskup
This is (probably) my fault. I played a bit with ipv6 && libvirt, and I
rebooted the machine, and it doesn't boot up now :-( sorry for the noise.
If there's some console (VPN?) access that we could use to fix such problems
within our (CPT) team?

There is currently no way to get to the consoles for these systems. That is
part of the bring up of that environment.

Post by Pavel Raiskup
Pavel

_______________________________________________
To unsubscribe send an email to
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
https://pagure.io/fedora-infrastructure

--
Stephen J Smoogen.

Stephen John Smoogen

2021-04-26 20:45:18 UTC

Permalink

Post by Stephen John Smoogen

There is currently no way to get to the consoles for these systems. That
is part of the bring up of that environment.

I will drive into the datacenter tomorrow to put a console on and try to
rescue the box. I will have to force a root login via USB key or similar.

Post by Stephen John Smoogen
Pavel

Post by Pavel Raiskup

--
Stephen J Smoogen.

Pavel Raiskup

2021-04-27 06:01:18 UTC

Permalink

Post by Stephen John Smoogen

There is currently no way to get to the consoles for these systems. That
is part of the bring up of that environment.

I will drive into the datacenter tomorrow to put a console on and try to
rescue the box. I will have to force a root login via USB key or similar.

Meh, sorry for complications, I'll try to be much more careful next time.

Perhaps you don't have to hurry up for the fix since we have the rest of
the stack (3 more boxes) that we still have to setup via ansible.. So
there's a chance that we'll break something else :-(.

I mean, because ipv4 addresses (32+ for each hypervisor) aren't available
I'm trying other things. Qemu port-forwarding to libvirt guests didn't
work for me (but I'll try better), ipv6 guest access did not work either
(I'll retry as well), so perhaps we'll have to try iptables hacks or
setup VPN(s) eventually. I'm afraid all those configuration tasks aren't
really that trivial to be 100% sure that no mistake happens :-(.

Pavel

Post by Stephen John Smoogen

Post by Stephen John Smoogen
Pavel

Post by Pavel Raiskup

--
Stephen J Smoogen.

Kevin Fenzi

2021-04-27 20:08:55 UTC

Permalink

Post by Pavel Raiskup
Meh, sorry for complications, I'll try to be much more careful next time.
Perhaps you don't have to hurry up for the fix since we have the rest of
the stack (3 more boxes) that we still have to setup via ansible.. So
there's a chance that we'll break something else :-(.
I mean, because ipv4 addresses (32+ for each hypervisor) aren't available
I'm trying other things. Qemu port-forwarding to libvirt guests didn't
work for me (but I'll try better), ipv6 guest access did not work either
(I'll retry as well), so perhaps we'll have to try iptables hacks or
setup VPN(s) eventually. I'm afraid all those configuration tasks aren't
really that trivial to be 100% sure that no mistake happens :-(.

I would think a bridge would work fine and just assign the guests ipv6
addresses? But I agree it will take some testing/tweaking. ;(

kevin

Pavel Raiskup

2021-04-28 08:42:46 UTC

Permalink

Post by Kevin Fenzi

I would think a bridge would work fine and just assign the guests ipv6
addresses? But I agree it will take some testing/tweaking. ;(

Indeed, I was able to configure one VM like that now - so I obviously did
something wrong before (I tested F33, now with F34, but I doubt this makes any
difference). So ... I think I shouldn't break the hypervisors anymore :-).

Pavel

_______________________________________________
infrastructure mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-***@lists.fedoraproject.org
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/***@lists.fedoraproject.org
Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructur