Difference between revisions of "CNMCyber Usable"

From CNM Wiki
Jump to: navigation, search
Line 93: Line 93:
  
 
===Database management===
 
===Database management===
: For the purposes of this wikipage, ''database management'' refers to those endeavors that are related to databases of the ''Cloud'':
+
: For the purposes of this wikipage, ''database management'' refers to those endeavors that are related to databases of the ''Cloud''.
:*'''[[MariaDB for CNM Farms]]'''. Further development of [[CNM MariaDB]] to explore using additional tools such as [[MariaDB MaxScale]] and [[MariaDB xPand]] in the existing MariaDB cluster of the ''Cloud''.
 
:*'''[[PostgreSQL for CNM Farms]]'''. New development of [[CNM PostgreSQL]] to create a cluster of [[PostgeSQL]] databases of the ''Cloud''. In general, linking similar databases with each other may make their functioning more stable.
 
:*'''[[DBs for Opplet]]'''. Further development of [[Opplet DB]] to optimize existing databases of [[Opplet]] and to add new federated databases that can be shared by [[CNM app]]s. New databases, for instance, can manage the data of the customers to be taken from and shared with [[CNM Venture]] (built with [[CNM Odoo]]) and [[CNM Linkup]] (built with [[CNM SuiteCRM]]). Earlier, the ''Cloud'' developers discussed possibilities of using [[MongoDB]] or [[Apache Cassandra]] for data storage, [[MuleESB]] for data collection and [[Apache Hadoop]] for "formating".
 
:*'''[[Ceph for CNM Farms]]'''. Further development of [[CNM Ceph]] to document and optimize it.
 
 
 
 
:{|class="wikitable" width=100% style="text-align:center;"
 
:{|class="wikitable" width=100% style="text-align:center;"
 
|+Authorized endeavors related to database management
 
|+Authorized endeavors related to database management
Line 132: Line 127:
 
|-
 
|-
 
|Software under consideration
 
|Software under consideration
![[]]!![[]]!![[]]!![[]]
+
![[MariaDB]]!![[PostgeSQL]]!![[Apache Cassandra]], [[MuleESB]], [[Apache Hadoop]]!![[Ceph]]
 
|}
 
|}
  
 
===DNS management===
 
===DNS management===
: For the purposes of this wikipage, ''DNS management'' refers to those endeavors that are utilized or can be used to manage [[DNS]] of the ''Cloud'':
+
: For the purposes of this wikipage, ''DNS management'' refers to those endeavors that are utilized or can be used to manage [[DNS]] of the ''Cloud''.
:*'''[[CDN for CNM Farms]]'''. New development of [[CNM Farm CDN]] to explore building one or more [[content delivery network]]s ([[CDN]]s) at least for some of [[CNM app]]s.
 
:*'''[[DNSSEC for CNM Farms]]'''. New development of [[CNM Farm DNSSEC]] to explore utilizing [[DNSSEC]] in the ''Cloud''.
 
:*'''[[Geocast for CNM Farms]]'''. New development of [[CNM Farm Geocast]] to explore adding [[Geocast]] or similar functionality to some part or parts of the ''Cloud''.
 
:*'''[[IPv6 for CNM Farms]]'''. Further development of [[CNM Farm IPv6]] to explore migration of the ''Cloud's'' DNS, except for public access, to [[IPv6]].
 
:*'''[[Revisions for CNM DNS]]'''. New development of [[CNM DNS revisions]] that aim to build the system of periodic revisions of all the ''Cloud's'' [[DNS record]]s.
 
 
 
 
:{|class="wikitable" width=100% style="text-align:center;"
 
:{|class="wikitable" width=100% style="text-align:center;"
 
|+Authorized endeavors related to DNS management
 
|+Authorized endeavors related to DNS management
Line 176: Line 165:
 
|-
 
|-
 
|Software under consideration
 
|Software under consideration
![[]]!![[]]!![[]]!![[]]
+
![[]]!![[]]!![[]]!!N/A
 
|}
 
|}
  
 
===Support tools===
 
===Support tools===
 
: For the purposes of this wikipage, ''support tools'' refer to those software development and sysadmin administrative tools that are (a) listed at https://github.com/kahun/awesome-sysadmin and (b) not mentioned in other sections of the endeavors.
 
: For the purposes of this wikipage, ''support tools'' refer to those software development and sysadmin administrative tools that are (a) listed at https://github.com/kahun/awesome-sysadmin and (b) not mentioned in other sections of the endeavors.
:* '''[[Panels for CNM Farms]]'''. New development of [[Panels in CNM Farms]] to explore adding control panels such as [[VestaCP]], as well as status systems such as [[Cachet]] to the ''Cloud''.
 
:* '''[[Workspaces for CNM Cloud]]'''. New development of [[Workspaces in CNM Cloud]] to explore adding personal tools and development environments such as [[Eclipse]] to the ''Cloud''.
 
:* '''[[Nextcloud for CNM Cloud]]'''. New development of [[CNM Nextcloud]] to explore adding [[Nextcloud]] as a set of personal management tools to the ''Cloud''.
 
 
 
:{|class="wikitable" width=100% style="text-align:center;"
 
:{|class="wikitable" width=100% style="text-align:center;"
 
|+Authorized endeavors related to support tools
 
|+Authorized endeavors related to support tools
Line 218: Line 203:
 
|-
 
|-
 
|Software under consideration
 
|Software under consideration
![[]]!![[]]!![[]]!![[]]
+
![[VestaCP]], [[Cachet]]!![[Eclipse]]!![[Nextcloud]]
 
|}
 
|}
  
 
===Node clusters===
 
===Node clusters===
 
: For the purposes of this wikipage, ''node clusters'' refers to clusters of nodes that are utilized or can be used to build the ''Farms''. The four ''Farms'' are made up of clustered nodes. Each cluster must have at least one (a) DNS entry point, which for highly available ''Farms'' includes a [[load balancer]] on a public web address, (b) synchronization of resources of common individual nodes, at least databases, (c) monitoring, ( d) security, including firewalls; and (e) backup and recovery systems.
 
: For the purposes of this wikipage, ''node clusters'' refers to clusters of nodes that are utilized or can be used to build the ''Farms''. The four ''Farms'' are made up of clustered nodes. Each cluster must have at least one (a) DNS entry point, which for highly available ''Farms'' includes a [[load balancer]] on a public web address, (b) synchronization of resources of common individual nodes, at least databases, (c) monitoring, ( d) security, including firewalls; and (e) backup and recovery systems.
:* '''[[Farm for Bureau]]'''. Further development of [[CNM Bureau Farm]] to advance its features. Currently, this ''Farm'' is based on two bare-metal servers. This ''Farm'' also utilizes a portion of one bare-metal server that belongs to [[CNM Lab Farm]]. The issues to work on may include (a) security outside of [[iptables]], (b) adding NAS, as well as advanced backup and recovery systems, and (c) advanced monitoring systems.
 
:* '''[[Farm for Campus]]'''. Further development of [[CNM Campus Farm]] to advance its features. Currently, this ''Farm'' is based on three [[virtual private server]]s ([[VPS]]es), which databases are synchronized via [[MariaDB Galera]], and high availability functionality is based on [[HAProxy]] and includes (a) entry point, (b) monitoring, (c) security, and (d) backup and recovery system. One of the servers is also connected to additional storage, which is supposed to be converted to NAS.
 
:* '''[[Farm for Lab]]'''. Further development of [[CNM Lab Farm]] to advance its features. Currently, this ''Farm'' is based on one bare-metal server. This ''Farm'' requires no high availability functionality due to the experimental nature of the applications installed on it. Due to the lack of high availability requirement, this farm will require an advanced backup and recovery system.
 
:* '''[[Farm for Opplet]]'''. Further development of [[Opplet Farm]] to advance its features. Currently, this ''Farm'' is based on two [[virtual private server]]s ([[VPS]]es); however, they haven't been included in one network. It is expected that a few developments of [[CNM Campus Farm]] will be used here.
 
 
: Previously, a private cloud built on [[OpenStack]] was used. It was closed due to high cost and low workload at the time. The development of the project may require the transfer of some of the Farms resources to the cloud solution again. Recently, [[Apache CloudStack]] has also become popular. If such a decision is made, it will be necessary to decide which support package to use and whether it will be the development of [[CNM Lab Farm]] or [[Opplet Farm]].
 
 
 
:{|class="wikitable" width=100% style="text-align:center;"
 
:{|class="wikitable" width=100% style="text-align:center;"
 
|+Authorized endeavors related to node clusters
 
|+Authorized endeavors related to node clusters
Line 263: Line 241:
 
|-
 
|-
 
|Software under consideration
 
|Software under consideration
![[]]!![[]]!![[]]!![[]]
+
![[ProxmoxVE]]!![[HAProxy]]!![[OpenStack]]!![[Apache CloudStack]]
 
|}
 
|}
  
Line 274: Line 252:
  
 
===App-oriented===
 
===App-oriented===
: For the purposes of this wikipage, ''app-oriented Opplet'' refers to those endeavors that are utilized or can be used on several rather than one individual Farm:
+
: For the purposes of this wikipage, ''app-oriented Opplet'' refers to those endeavors to build [[Opplet]] that support [[CNM app]]s rather than its [[end-user]]s.
:*'''[[WSO2 IS for Opplet]]'''. New development of [[Opplet WSO2 IS]] to add [[WSO2 Identity Server]] ([[WSO2 Identity Server|WSO2 IS]]) services to [[Opplet]]. Currently, ''Opplet'' utilizes [[OpenLDAP]] as its authorization and identity connector. However, ''OpenLDAP'' does not allow [[single sign-on]] ([[SSO]]) services. In addition, the ''Opplet'' developers faced troubles in changing roles and in connecting some end-user applications such as [[MediaWiki]] with ''OpenLDAP''. Adding ''WSO2 IS'' also aims to introduce user registration in ''Opplet'' within [[CNM app]]s. Currently, the users must first register with ''Opplet'' in order to use those [[CNM app]]'s services that are available for the registered users only. It is unclear at the moment whether ''OpenLDAP'' shall be kept in ''Opplet'' after ''WSO2 IS'' addition and, if so, how.
 
:*'''[[Enrollment to Opplet]]'''. New development of [[Opplet Enrollment]] to transfer the course registration (which [[Moodle]] in Australian way calls "enrolment") from [[CNM Cert]] to [[Opplet]]. Beyond visibility of user enrollment status on Opplet dashboard, this transfer shall resolve several problems: (a) speed. The current enrollment is carried out through the cron tool, which has a response delay, (b) enrollment in [[LMS]]es other than ''CNM Cert''. The ''Cert'' accommodates only first 3 levels of the educational programs, (c) automatic unenrollment from those courses that have successfully been completed.
 
:*'''[[IAM to Opplet]]'''. Further development of [[Opplet IAM]] to (a) automatically add new federated [[Opplet role]]s based on the course progress and (b) communicate those roles to [[CNM app]]s. Currently, ''Opplet'' user roles can only be changed manually by ''Opplet'' administrators.
 
:*'''[[Mail for CNM Cloud]]'''. Further development of [[CNM Mail Server]] to make sure that all [[CNM app]]s can send and, when applicable, receive emails through it.
 
:*'''[[Exams to Opplet]]'''. New development of [[Opplet Exams]] to move test banks from [[CNM Moodle]] to ''Opplet''.
 
:*'''[[Warehouse for CNM Cloud]]'''. New development of [[Opplet Warehouse]] to create a federated object storage in order to make illustrations and other files available for various [[CNM app]]s. The ''Cloud'' developers earlier also discussed an idea was to utilize [[CNM GitLab]] as a file storage.
 
 
 
 
:{|class="wikitable" width=100% style="text-align:center;"
 
:{|class="wikitable" width=100% style="text-align:center;"
 
|+Authorized endeavors related to app-oriented Opplet
 
|+Authorized endeavors related to app-oriented Opplet
Line 315: Line 286:
 
|-
 
|-
 
|Software under consideration
 
|Software under consideration
![[]]!![[]]!![[]]!![[]]
+
![[WSO2 IS]]!![[Yii]]!![[Yii]]!![[]]!![[Yii]]!![[]]
 
|}
 
|}
  
 
===User-oriented===
 
===User-oriented===
: For the purposes of this wikipage, ''user-oriented Opplet'' refers to those services that [[Opplet]] provides to its end-users:
+
: For the purposes of this wikipage, ''user-oriented Opplet'' refers to those endeavors to build [[Opplet]] that support its [[end-user]]s rather than [[CNM app]]s.
:*'''[[Interface for Opplet]]'''. Further development of [[Opplet interface]] to improve its usability and graphic design.
 
:*'''[[Events for Opplet]]'''. New development of [[Opplet Events]] to (a) organize events on the administrative side and (b) manage own participation on the end-user side in the ''Cloud''.
 
:*'''[[Mailboxes to Opplet]]'''. Further development of [[Opplet Mailboxes]] to accommodate its automatic creation, management, and cancelation in order to upgrade this feature from its [[minimum viable product]] ([[minimum viable product|MVP]]) state up to the [[marketable]] one.
 
:*'''[[Newsletters to Opplet]]'''. New development of [[Opplet Newsletters]] to add mailing list and subscription functionalities.
 
:*'''[[Calendar for Opplet]]'''. New development of [[Opplet Calendar]] to add a federated calendar to [[Opplet]].
 
:*'''[[CNM Registration Project]]'''. Further development of [[Opplet profile]]s.
 
 
 
 
:{|class="wikitable" width=100% style="text-align:center;"
 
:{|class="wikitable" width=100% style="text-align:center;"
 
|+Authorized endeavors related to user-oriented Opplet
 
|+Authorized endeavors related to user-oriented Opplet

Revision as of 15:13, 9 December 2022

CNM Cloud Usable (in the Russian-language version, known as Работы над Облаком) is the fourth, current, phase of the CNM Cloud Project. CNM Cloud (hereinafter - the Cloud) is a set of technologies that supports CNM Cyber. This very wikipage presents the primary list of those endeavors that are included in that phase. CNM Cyber Project Coordinators (hereinafter, the Coordinators) are invited to organize work on its endeavors.


Trivia

Projects vs operations

The endeavors include:
  • Projects undertaken to develop new systems to be deployed at the Cloud.
  • Operations undertaken to sustain productive functions of deployed systems of the Cloud.

Cloud layers

  1. CNM Farms. The Cloud infrastructure or, in other words, ecosystem of computing servers and cloud OS to support both Opplet and CNM apps.
  2. Opplet. The enterprise-wide utility software that supports CNM apps regardless of their location of those applications in the Cloud.
  3. CNM apps. End-user applications of the Cloud.

Farms

CNM Farms (hereinafter, the Farms) present the infrastructure for the functioning of both the CNM apps and Opplet that serves those apps. The Farms are assembled using commercial off-the-shelf (COTS) software only.

Farms' administration

For the purposes of this wikipage, the Farms' administration refers to those endeavors that are utilized or can be used on several rather than one individual Farm.
Authorized endeavors related to the Farms' administration
Progress Integration Configuration Statistics Revisions (farms)
Business requirements Done      
Prototypes        
Stakeholder requirements        
Deliverable specifications        
Work specifications        
Certainty        
Utility        
Applicability        
Controllability        
Software under consideration Jenkins, Kafka, OpenVPN Terraform, Ansible ZooKeeper N/A

Farms' monitoring

For the purposes of this wikipage, the Farms' monitoring refers to those systems that monitor one or more Farms.
Authorized endeavors related to the Farms' monitoring
Progress Grafana (work) Zabbix (work) Nagios (work)
Business requirements      
Prototypes      
Stakeholder requirements      
Deliverable specifications      
Work specifications      
Certainty      
Utility      
Applicability      
Controllability      
Software under consideration Grafana Zabbix Nagios

Database management

For the purposes of this wikipage, database management refers to those endeavors that are related to databases of the Cloud.
Authorized endeavors related to database management
Работы MariaDB (work) PostgreSQL (work) Opplet (DBs) Ceph (work)
Business requirements        
Prototypes        
Stakeholder requirements        
Deliverable specifications        
Work specifications        
Certainty        
Utility        
Applicability        
Controllability        
Software under consideration MariaDB PostgeSQL Apache Cassandra, MuleESB, Apache Hadoop Ceph

DNS management

For the purposes of this wikipage, DNS management refers to those endeavors that are utilized or can be used to manage DNS of the Cloud.
Authorized endeavors related to DNS management
Progress CDN DNSSEC Geocast IPv6 Revisions (DNS)
Business requirements          
Prototypes          
Stakeholder requirements          
Deliverable specifications          
Work specifications          
Certainty          
Utility          
Applicability          
Controllability          
Software under consideration [[]] [[]] [[]] N/A

Support tools

For the purposes of this wikipage, support tools refer to those software development and sysadmin administrative tools that are (a) listed at https://github.com/kahun/awesome-sysadmin and (b) not mentioned in other sections of the endeavors.
Authorized endeavors related to support tools
Progress Panels Workspaces Nextcloud (work)
Business requirements      
Prototypes      
Stakeholder requirements      
Deliverable specifications      
Work specifications      
Certainty      
Utility      
Applicability      
Controllability      
Software under consideration VestaCP, Cachet Eclipse Nextcloud

Node clusters

For the purposes of this wikipage, node clusters refers to clusters of nodes that are utilized or can be used to build the Farms. The four Farms are made up of clustered nodes. Each cluster must have at least one (a) DNS entry point, which for highly available Farms includes a load balancer on a public web address, (b) synchronization of resources of common individual nodes, at least databases, (c) monitoring, ( d) security, including firewalls; and (e) backup and recovery systems.
Authorized endeavors related to node clusters
Progress Bureau (farm) Campus (farm) Lab (farm) Opplet (farm)
Business requirements        
Prototypes        
Stakeholder requirements        
Deliverable specifications        
Work specifications        
Certainty        
Utility        
Applicability        
Controllability        
Software under consideration ProxmoxVE HAProxy OpenStack Apache CloudStack

Opplet

Opplet represents (a) enterprise-wide middleware that serves the Cloud and (b) a group of end-user applications that allows end-users to utilize it. Therefore, the Opplet's functionalities can be divided into two groups:

  1. Services for CNM apps. Those federation services that Opplet provides to CNM apps.
  2. Services for end-users. Those services that Opplet provides to its end-users.

Some endeavors that are related to Opplet are included in the Farm for Opplet project (section Node clusters of this very wikipage) and DBs for Opplet project (section Database management). The core of Opplet is written specifically for the Cloud using Yii framework. Its periphery devices are assembled using commercial off-the-shelf (COTS) software.

App-oriented

For the purposes of this wikipage, app-oriented Opplet refers to those endeavors to build Opplet that support CNM apps rather than its end-users.
Authorized endeavors related to app-oriented Opplet
Progress WSO2 IS (work) Enrollment IAM Mail (Cloud) Exams Warehouse
Business requirements            
Prototypes            
Stakeholder requirements            
Deliverable specifications            
Work specifications            
Certainty            
Utility            
Applicability            
Controllability            
Software under consideration WSO2 IS Yii Yii [[]] Yii [[]]

User-oriented

For the purposes of this wikipage, user-oriented Opplet refers to those endeavors to build Opplet that support its end-users rather than CNM apps.
Authorized endeavors related to user-oriented Opplet
Progress Interface (Opplet) Events (Opplet) Mailboxes (Opplet) Newsletters (Opplet) Calendar (Opplet) Registration
Business requirements            
Prototypes            
Stakeholder requirements            
Deliverable specifications            
Work specifications            
Certainty            
Utility            
Applicability            
Controllability            
Software under consideration [[]] [[]] [[]] [[]]

CNM apps

CNM apps (hereinafter, the Apps) present end-user applications that are available to users of the Cloud. The Apps include:

  1. CNM stable apps that have at least 2 installations -- one is used for its target purposes and another is available for hands-on training. Any stable App needs to be able to work with our future Opplet WSO2 IS and, possibly, existing OpenLDAP. The Apps are assembled using commercial off-the-shelf (COTS) software only. Those endeavors that need to be undertaken for every stable App, must encompass (a) updating the Apps to the latest stable versions and install the latest patches if and when they become available and (b) documenting what we have and identify problems.
  2. CNM future apps that are installed on the CNM Lab Farm only. These apps represent either popular COTS packages or end-user application that can be deployed in the Cloud as CNM stable apps in the future.

This section lists both (a) those Apps that are incorporated, both CNM stable and future apps, as well as (b) some applications that may be potentially incorporated in the Cloud. Some endeavors that are related to the Apps are included in projects listed in DNS management and App-oriented sections of this very wikipage.

Utilizing MariaDB

For the purposes of this wikipage, utilizing MariaDB apps refer to those CNM stable apps that utilize MariaDB as its database management system. The Cloud incorporates at least five of them.
Authorized endeavors related to utilizing MariaDB apps
Progress AVideo (work) WordPress (work) MediaWiki (work) HumHub (work) Moodle (work)
Business requirements          
Prototypes          
Stakeholder requirements          
Deliverable specifications          
Work specifications          
Certainty          
Utility          
Applicability          
Controllability          
Software under consideration [[]] [[]] [[]] [[]]

Utilizing non-MariaDB

For the purposes of this wikipage, utilizing non-MariaDB apps refer to those CNM stable apps that don't utilize MariaDB as its database management system. The Cloud incorporates at least four of those Apps:
Authorized endeavors related to utilizing non-MariaDB apps
Progress GitLab (work) SuiteCRM (work) Odoo (work) Jitsi (work)
Business requirements        
Prototypes        
Stakeholder requirements        
Deliverable specifications        
Work specifications        
Certainty        
Utility        
Applicability        
Controllability        
Software under consideration [[]] [[]] [[]] [[]]

Experimental

For the purposes of this wikipage, experimental apps refer to (a) CNM future apps, (b) some applications that have been installed to be analyzed and evaluated, (c) those applications that are to be utilized during short hands-on training sessions, and (d) legacy applications that are retired from CNM stable app, but are still kept as "museum" exhibits.
Authorized endeavors related to experimental apps
Progress OpenEdX (work) ProjecQtOr (work) Redmine (work) Taiga (work)
Business requirements        
Prototypes        
Stakeholder requirements        
Deliverable specifications        
Work specifications        
Certainty        
Utility        
Applicability        
Controllability        
Software under consideration [[]] [[]] [[]] [[]]

Enterprise

Venture administration

For the purposes of this wikipage, venture administration refers to human administration of the Cloud enterprise.
Authorized endeavors related to enterprise administration
Progress Personnel Agency Finance Legal
Business requirements        
Prototypes        
Stakeholder requirements        
Deliverable specifications        
Work specifications        
Certainty        
Utility        
Applicability        
Controllability        
Software under consideration [[]] [[]] [[]] [[]]

External operations

For the purposes of this wikipage, enterprise external operations refer to those endeavors that are utilized or can be used outside of the enterprise behind the Cloud.
Authorized endeavors related to enterprise external operations
Progress Industry Developers Technologies Help Desk Operators
Business requirements          
Prototypes          
Stakeholder requirements          
Deliverable specifications          
Work specifications          
Certainty          
Utility          
Applicability          
Controllability          
Software under consideration [[]] [[]] [[]] [[]]

Stakeholder groups

For the purposes of this wikipage, stakeholder groups refer to those endeavors that are utilized or can be used to manage groups of the Cloud stakeholders:
Authorized endeavors related to stakeholder groups
Progress Councils Committees Users Sysadmins
Business requirements        
Prototypes        
Stakeholder requirements        
Deliverable specifications        
Work specifications        
Certainty        
Utility        
Applicability        
Controllability        
Software under consideration [[]] [[]] [[]] [[]]

Name yours