Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[reconfigurator] Move resource allocation out of BlueprintBuilder #7235

Open
wants to merge 14 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 12 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
392 changes: 256 additions & 136 deletions nexus/reconfigurator/planning/src/blueprint_builder/builder.rs

Large diffs are not rendered by default.

2 changes: 0 additions & 2 deletions nexus/reconfigurator/planning/src/blueprint_builder/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,6 @@

mod builder;
mod clickhouse;
mod external_networking;
mod internal_dns;

pub use builder::*;
pub use clickhouse::{ClickhouseAllocator, ClickhouseZonesThatShouldBeRunning};
22 changes: 21 additions & 1 deletion nexus/reconfigurator/planning/src/blueprint_editor.rs
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,29 @@
//!
//! See crate-level documentation for details.

mod allocators;
mod sled_editor;

pub use allocators::BlueprintResourceAllocatorInputError;
pub use allocators::ExternalNetworkingError;
pub use allocators::InternalDnsError;
pub use allocators::InternalDnsInputError;
pub use sled_editor::DatasetsEditError;
pub use sled_editor::DisksEditError;
pub use sled_editor::DuplicateDiskId;
pub use sled_editor::DuplicateZoneId;
pub use sled_editor::MultipleDatasetsOfKind;
pub use sled_editor::SledEditError;
pub use sled_editor::SledInputError;
pub use sled_editor::SledUnderlayIpOutOfRange;
pub use sled_editor::ZonesEditError;

pub(crate) use allocators::BlueprintResourceAllocator;
pub(crate) use allocators::ExternalNetworkingChoice;
pub(crate) use allocators::ExternalSnatNetworkingChoice;
pub(crate) use sled_editor::DatasetIdsBackfillFromDb;
pub(crate) use sled_editor::EditedSled;
pub(crate) use sled_editor::SledEditError;
pub(crate) use sled_editor::SledEditor;

#[cfg(test)]
pub(crate) use allocators::ExternalIpAllocator;
114 changes: 114 additions & 0 deletions nexus/reconfigurator/planning/src/blueprint_editor/allocators.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,114 @@
// This Source Code Form is subject to the terms of the Mozilla Public
// License, v. 2.0. If a copy of the MPL was not distributed with this
// file, You can obtain one at https://mozilla.org/MPL/2.0/.

//! Blueprint planner resource allocation
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you planning to add other resource allocation besides networking stuff here? I'm asking because the name is pretty generic, but I assume that is on purpose.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I thought we could (e.g.) move ClickhouseAllocator under this module at some point as we rework the builder. I'd also like zpool allocation to land here (it's currently just a method on BlueprintBuilder) eventually.


use std::net::IpAddr;

use super::SledEditor;
use nexus_types::deployment::BlueprintZoneFilter;
use omicron_common::address::DnsSubnet;
use omicron_common::address::IpRange;
use omicron_common::address::ReservedRackSubnet;
use omicron_uuid_kinds::SledUuid;

mod external_networking;
mod internal_dns;

pub use self::external_networking::ExternalNetworkingError;
pub use self::internal_dns::InternalDnsError;
pub use self::internal_dns::InternalDnsInputError;

pub(crate) use self::external_networking::ExternalNetworkingChoice;
pub(crate) use self::external_networking::ExternalSnatNetworkingChoice;

#[cfg(test)]
pub(crate) use self::external_networking::ExternalIpAllocator;

use self::external_networking::ExternalNetworkingAllocator;
use self::internal_dns::InternalDnsSubnetAllocator;

#[derive(Debug, thiserror::Error)]
pub enum BlueprintResourceAllocatorInputError {
#[error(transparent)]
InternalDns(#[from] InternalDnsInputError),
#[error("failed to create external networking allocator")]
ExternalNetworking(#[source] anyhow::Error),
}

#[derive(Debug)]
pub(crate) struct BlueprintResourceAllocator {
external_networking: ExternalNetworkingAllocator,
internal_dns: InternalDnsSubnetAllocator,
}

impl BlueprintResourceAllocator {
pub fn new<'a, I>(
all_sleds: I,
service_ip_pool_ranges: Vec<IpRange>,
target_internal_dns_redundancy: usize,
) -> Result<Self, BlueprintResourceAllocatorInputError>
where
I: Iterator<Item = (SledUuid, &'a SledEditor)> + Clone,
{
let internal_dns = InternalDnsSubnetAllocator::new(
all_sleds.clone().flat_map(|(_, editor)| {
editor.zones(BlueprintZoneFilter::ShouldBeRunning)
}),
target_internal_dns_redundancy,
)?;

let external_networking = ExternalNetworkingAllocator::new(
all_sleds.clone().flat_map(|(_, editor)| {
editor.zones(BlueprintZoneFilter::ShouldBeRunning)
}),
all_sleds.flat_map(|(_, editor)| {
editor.zones(BlueprintZoneFilter::Expunged)
}),
service_ip_pool_ranges,
)
.map_err(BlueprintResourceAllocatorInputError::ExternalNetworking)?;

Ok(Self { external_networking, internal_dns })
}

pub fn next_internal_dns_subnet(
&mut self,
rack_subnet: ReservedRackSubnet,
) -> Result<DnsSubnet, InternalDnsError> {
self.internal_dns.alloc(rack_subnet)
}

pub(crate) fn next_external_ip_nexus(
&mut self,
) -> Result<ExternalNetworkingChoice, ExternalNetworkingError> {
self.external_networking.for_new_nexus()
}

pub(crate) fn next_external_ip_external_dns(
&mut self,
) -> Result<ExternalNetworkingChoice, ExternalNetworkingError> {
self.external_networking.for_new_external_dns()
}

pub(crate) fn next_external_ip_boundary_ntp(
&mut self,
) -> Result<ExternalSnatNetworkingChoice, ExternalNetworkingError> {
self.external_networking.for_new_boundary_ntp()
}

/// Allow a test to manually add an external DNS address, which could
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could probably add a testing feature for this, but it's likely not worth it. I really wish rust let stuff marked with #[cfg(test)] be used in tests outside the crate.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed on both counts.

/// ordinarily only come from RSS.
///
/// TODO-cleanup: Remove when external DNS addresses are in the policy.
// This can't be `#[cfg(test)]` because it's used by the `ExampleSystem`
// helper (which itself is used by reconfigurator-cli and friends). We give
// it a scary name instead.
pub(crate) fn inject_untracked_external_dns_ip(
&mut self,
ip: IpAddr,
) -> Result<(), ExternalNetworkingError> {
self.external_networking.add_external_dns_ip(ip)
}
}
Loading
Loading