Difference between revisions of "CNMC bare-metal"
(→App-level HA) |
|||
| (47 intermediate revisions by the same user not shown) | |||
| Line 1: | Line 1: | ||
| − | The [[CNMC bare-metal]] (hereinafter, the '' | + | The [[CNMC bare-metal]] (hereinafter, the ''Metals'') is the technology that supports the [[CNMCyber]] project, [[Urdu Project]], and [[CNMCyber]]'s project in Ukraine. |
==Needs== | ==Needs== | ||
| − | The '' | + | The ''Metals<nowiki>'</nowiki>'' owner would like to be in business of workforce development. The ''Metals'' shall provide learners, who are employment aspirants, with laboratory environment that would include various end-user applications, as well as virtual machines (hereinafter, ''VM''s) and containers that host them. The end-user applications shall include [[AVideo]], [[GitLab]], [[HumHub]], [[MediaWiki]], [[Moodle]], [[Odoo]], [[Roundcube]], [[SuiteCRM]], and [[WordPress]] and some others. Through their hands-on training and internships, the learners shall possess those competencies and credentials that shall help these learners to land new jobs or advance in their current employment. These competencies and credentials shall range from bare-metal server to end-user application design, implementation, administration, and retirement. |
===User stories=== | ===User stories=== | ||
| Line 9: | Line 9: | ||
#As a training provider, I want to have authority to give a particular learner to access a ''VM'' that hosts a particular software package, so this learner will be able to explore, provision, modify, and restore the package. | #As a training provider, I want to have authority to give a particular learner to access a ''VM'' that hosts a particular software package, so this learner will be able to explore, provision, modify, and restore the package. | ||
#As a learner, I want to access a ''VM'', which I am authorized to access, in order to explore, provision, modify, and restore the application that this ''VM'' hosts. | #As a learner, I want to access a ''VM'', which I am authorized to access, in order to explore, provision, modify, and restore the application that this ''VM'' hosts. | ||
| + | #As an employment candidate, I want to show my skills and experience of working with software during employment interviews. | ||
===Milestones=== | ===Milestones=== | ||
We plan to: | We plan to: | ||
| − | # | + | #Re-design the ''Metals'', [[RAID]], [[OpenZFS]] and [[ProxmoxVE]], as well as secure them. |
| − | #Setup a high-availability secure cluster of [[WordPress]] instances on the | + | #Setup a high-availability secure cluster of [[WordPress]] instances on the ''Metals''. |
| − | # | + | #Explore high-availability and Geocast options for the instances installed on the ''Metals'' and VPSes. |
==Server layer== | ==Server layer== | ||
| − | To support the project | + | To support the project technologically, we created a platform with 3 VPS at contabo.com and one proxmox instance at hetzner.de in Germany. At the nest stage of the project, we may add one or two bare-metal servers in the same datacenter with the existing one. We would like to explore how to test the purchased server or servers. The additional server or servers may or may not be similar to the current one, probably, with smaller RAM. Later, we would also like to explore OVH servers. |
| − | === | + | ===Virtualization=== |
| − | The characteristics of the server | + | [[ProxmoxVE]] is the only package under consideration; [[Debian]] shall be used as its [[OS]]. |
| + | |||
| + | ===Server security=== | ||
| + | We plan 5 measures to ensure the ''Metals'' security at the server layer: | ||
| + | #Using iptables, ban everything what is not allowed | ||
| + | #fail2ban to prevent brute-force attacks | ||
| + | #Changing default ports, especially 22/SSH | ||
| + | #Setting up the new SSH port to key based authentication so who have the key they can access. | ||
| + | #Creating a whitelist of static IP addresses that are allowed to access the new SSH port | ||
| + | One specialist also proposed to use Config Server Security and Firewall (CSF), so when anyone do wrong attempt with our server it will auto blocked on three wrong attempt, also we can block any country any location any ip any isp etc with firewall. | ||
| + | |||
| + | ===Server-level HA=== | ||
| + | [[ProxmoxVE]] ships with its HA tool called [[HA-manager]]; however, it requires 3 bare-metal servers to be implemented. At this stage, we consider working with only one bare-metal server. Thus, we consider using [[HAProxy]] and/or, possibly, [[CDN]] between the existing bare-metal server and VPSes. | ||
| + | |||
| + | ==Metal 1== | ||
| + | |||
| + | The characteristics of the first bare-metal server are as follows: | ||
*Dedicated Root Server SB35 | *Dedicated Root Server SB35 | ||
*Intel Core i7-3930 | *Intel Core i7-3930 | ||
| Line 28: | Line 45: | ||
*Location: FSN1 (Falkenstein/Vogtland, Germany) -- DC7 | *Location: FSN1 (Falkenstein/Vogtland, Germany) -- DC7 | ||
*Rescue system (English) | *Rescue system (English) | ||
| − | + | One expert stated that using i7 processors have some disadvantages. Another expert replied that, indeed, i7 processors may have troubles with something like [[PCI passthrough]], but for this particular project at this particular stage, they are just fine. | |
| + | |||
| + | ===Disk-redundancy 1=== | ||
| + | [[ProxmoxVE]] ships with [[OpenZFS]], so, we consider using it strongly. Other implementations of software-based [[RAID]] at various levels shall also be discussed. We don't consider hardware-based solutions at this stage to avoid dependency on proprietary RAID cards. | ||
| + | |||
| + | ===IP addresses 1=== | ||
| + | On the main server, either local IP or private IP range with DHCP is used. Unless specific concerns arise, we plan to use 2 IPv4 addresses. We are also open to explore IPv6. If we use About ipv4 addresses, we need 5 ips: one for main server, two for gateway, three for any vps or container and four for wordpress vps, and 5 for anyother we need in future. | ||
| − | + | [[OpenVPN]] | |
| + | |||
| + | ===Server monitoring 1=== | ||
| + | To monitor the ''Metals'' performance at the server layer, we plan to deploy both: | ||
| + | *[[Zabbix]] | ||
| + | *[[Nagios]] | ||
| − | === | + | ==Metal 2== |
| − | |||
| − | + | The characteristics of the second bare-metal server are as follows: | |
| − | + | *Dedicated Root Server | |
| + | * Intel Xeon E3-1245 | ||
| + | * 2x HDD SATA 3,0 TB Enterprise | ||
| + | * 4x RAM 8192 MB DDR3 ECC | ||
| + | * NIC 1 Gbit Intel 82574L | ||
| + | * RAID Controller 4-Port SATA PCI-E LSI MegaRAID SAS 9260-4i | ||
| + | * Location: FSN1 (Falkenstein/Vogtland, Germany) -- DC7 | ||
| + | * Rescue system (English) | ||
| − | === | + | ===Disk-redundancy 2=== |
| − | + | [[ProxmoxVE]] ships with [[OpenZFS]], so, we consider using it strongly. Other implementations of software-based [[RAID]] at various levels shall also be discussed. We don't consider hardware-based solutions at this stage to avoid dependency on proprietary RAID cards. | |
| − | |||
| − | |||
| − | ===IP addresses=== | + | ===IP addresses 2=== |
On the main server, either local IP or private IP range with DHCP is used. Unless specific concerns arise, we plan to use 2 IPv4 addresses. We are also open to explore IPv6. If we use About ipv4 addresses, we need 5 ips: one for main server, two for gateway, three for any vps or container and four for wordpress vps, and 5 for anyother we need in future. | On the main server, either local IP or private IP range with DHCP is used. Unless specific concerns arise, we plan to use 2 IPv4 addresses. We are also open to explore IPv6. If we use About ipv4 addresses, we need 5 ips: one for main server, two for gateway, three for any vps or container and four for wordpress vps, and 5 for anyother we need in future. | ||
| − | + | [[OpenVPN]] | |
| − | |||
| − | ===Server | + | ===Server monitoring 2=== |
| − | + | To monitor the ''Metals'' performance at the server layer, we plan to deploy both: | |
| + | *[[Zabbix]] | ||
| + | *[[Nagios]] | ||
==Application layer== | ==Application layer== | ||
| − | + | Our main server and its WordPress instances were hacked a few times. This is a description of what happened with the server (it is in Russian though) -- https://pravka.bskol.com/ru/%D0%9E%D0%BF%D1%8B%D1%82%D0%BD%D1%8B_%D0%9F%D1%80%D0%BE%D0%B5%D0%BA%D1%82 | |
| + | |||
| + | ===HA clusters=== | ||
| + | We plan to use [https://pve.proxmox.com/wiki/High_Availability_Cluster High_Availability_Cluster] | ||
| + | |||
| + | In addition to the bare-server, we plan to deploy one VPS in the US to extend the high-availability of its core applications. We cannot consider Cloudflare for the project because this would limit hands-on training opportunities for future students. | ||
| + | |||
| + | ===Monitoring=== | ||
| + | |||
| + | ===Firewalls=== | ||
| + | |||
| + | ==End-user applications== | ||
| + | We would like to start with Wordpress. Then, we add MediaWiki, Moodle and the rest. | ||
===WordPress=== | ===WordPress=== | ||
| Line 60: | Line 105: | ||
*WordPress -- VM or container model? we can use kvm based vms as well, but normally containers will be ok. we can use one vps for wordpress also. | *WordPress -- VM or container model? we can use kvm based vms as well, but normally containers will be ok. we can use one vps for wordpress also. | ||
| − | *What would we do for its security? For securety we can use antivirus | + | *What would we do for its security? For securety we can use antivirus |
| − | + | ===MediaWiki=== | |
| − | === | + | ===Moodle=== |
| − | |||
| − | + | ==Development== | |
Latest revision as of 19:10, 15 March 2023
The CNMC bare-metal (hereinafter, the Metals) is the technology that supports the CNMCyber project, Urdu Project, and CNMCyber's project in Ukraine.
Contents
Needs
The Metals' owner would like to be in business of workforce development. The Metals shall provide learners, who are employment aspirants, with laboratory environment that would include various end-user applications, as well as virtual machines (hereinafter, VMs) and containers that host them. The end-user applications shall include AVideo, GitLab, HumHub, MediaWiki, Moodle, Odoo, Roundcube, SuiteCRM, and WordPress and some others. Through their hands-on training and internships, the learners shall possess those competencies and credentials that shall help these learners to land new jobs or advance in their current employment. These competencies and credentials shall range from bare-metal server to end-user application design, implementation, administration, and retirement.
User stories
- As a workforce developer, I need my learners to have hands-on training opportunities on those software packages that are in demand on the job market.
- As a training provider, I want to have authority to give a particular learner to access a VM that hosts a particular software package, so this learner will be able to explore, provision, modify, and restore the package.
- As a learner, I want to access a VM, which I am authorized to access, in order to explore, provision, modify, and restore the application that this VM hosts.
- As an employment candidate, I want to show my skills and experience of working with software during employment interviews.
Milestones
We plan to:
- Re-design the Metals, RAID, OpenZFS and ProxmoxVE, as well as secure them.
- Setup a high-availability secure cluster of WordPress instances on the Metals.
- Explore high-availability and Geocast options for the instances installed on the Metals and VPSes.
Server layer
To support the project technologically, we created a platform with 3 VPS at contabo.com and one proxmox instance at hetzner.de in Germany. At the nest stage of the project, we may add one or two bare-metal servers in the same datacenter with the existing one. We would like to explore how to test the purchased server or servers. The additional server or servers may or may not be similar to the current one, probably, with smaller RAM. Later, we would also like to explore OVH servers.
Virtualization
ProxmoxVE is the only package under consideration; Debian shall be used as its OS.
Server security
We plan 5 measures to ensure the Metals security at the server layer:
- Using iptables, ban everything what is not allowed
- fail2ban to prevent brute-force attacks
- Changing default ports, especially 22/SSH
- Setting up the new SSH port to key based authentication so who have the key they can access.
- Creating a whitelist of static IP addresses that are allowed to access the new SSH port
One specialist also proposed to use Config Server Security and Firewall (CSF), so when anyone do wrong attempt with our server it will auto blocked on three wrong attempt, also we can block any country any location any ip any isp etc with firewall.
Server-level HA
ProxmoxVE ships with its HA tool called HA-manager; however, it requires 3 bare-metal servers to be implemented. At this stage, we consider working with only one bare-metal server. Thus, we consider using HAProxy and/or, possibly, CDN between the existing bare-metal server and VPSes.
Metal 1
The characteristics of the first bare-metal server are as follows:
- Dedicated Root Server SB35
- Intel Core i7-3930
- 2x HDD SATA 3,0 TB
- 8x RAM 8192 MB DDR3
- NIC 1 Gbit - Intel 82579LM
- Location: FSN1 (Falkenstein/Vogtland, Germany) -- DC7
- Rescue system (English)
One expert stated that using i7 processors have some disadvantages. Another expert replied that, indeed, i7 processors may have troubles with something like PCI passthrough, but for this particular project at this particular stage, they are just fine.
Disk-redundancy 1
ProxmoxVE ships with OpenZFS, so, we consider using it strongly. Other implementations of software-based RAID at various levels shall also be discussed. We don't consider hardware-based solutions at this stage to avoid dependency on proprietary RAID cards.
IP addresses 1
On the main server, either local IP or private IP range with DHCP is used. Unless specific concerns arise, we plan to use 2 IPv4 addresses. We are also open to explore IPv6. If we use About ipv4 addresses, we need 5 ips: one for main server, two for gateway, three for any vps or container and four for wordpress vps, and 5 for anyother we need in future.
Server monitoring 1
To monitor the Metals performance at the server layer, we plan to deploy both:
Metal 2
The characteristics of the second bare-metal server are as follows:
- Dedicated Root Server
- Intel Xeon E3-1245
- 2x HDD SATA 3,0 TB Enterprise
- 4x RAM 8192 MB DDR3 ECC
- NIC 1 Gbit Intel 82574L
- RAID Controller 4-Port SATA PCI-E LSI MegaRAID SAS 9260-4i
- Location: FSN1 (Falkenstein/Vogtland, Germany) -- DC7
- Rescue system (English)
Disk-redundancy 2
ProxmoxVE ships with OpenZFS, so, we consider using it strongly. Other implementations of software-based RAID at various levels shall also be discussed. We don't consider hardware-based solutions at this stage to avoid dependency on proprietary RAID cards.
IP addresses 2
On the main server, either local IP or private IP range with DHCP is used. Unless specific concerns arise, we plan to use 2 IPv4 addresses. We are also open to explore IPv6. If we use About ipv4 addresses, we need 5 ips: one for main server, two for gateway, three for any vps or container and four for wordpress vps, and 5 for anyother we need in future.
Server monitoring 2
To monitor the Metals performance at the server layer, we plan to deploy both:
Application layer
Our main server and its WordPress instances were hacked a few times. This is a description of what happened with the server (it is in Russian though) -- https://pravka.bskol.com/ru/%D0%9E%D0%BF%D1%8B%D1%82%D0%BD%D1%8B_%D0%9F%D1%80%D0%BE%D0%B5%D0%BA%D1%82
HA clusters
We plan to use High_Availability_Cluster
In addition to the bare-server, we plan to deploy one VPS in the US to extend the high-availability of its core applications. We cannot consider Cloudflare for the project because this would limit hands-on training opportunities for future students.
Monitoring
Firewalls
End-user applications
We would like to start with Wordpress. Then, we add MediaWiki, Moodle and the rest.
WordPress
core, plugin, theme cloudflare as web app firewall
- WordPress -- VM or container model? we can use kvm based vms as well, but normally containers will be ok. we can use one vps for wordpress also.
- What would we do for its security? For securety we can use antivirus
