运维123 - harbor
type: channel
link: https://www.sddts.cn/index.php/category/harbor/
items: 运维用容器镜像仓库
items: harbor - 简单做了一个私域容器镜像仓库
items: arm aarch x86_64这些属于啥，关系是啥，还有其他的架构么？
items: 镜像dockerhub到本地harbor-20231114
items: 清理并部署harbor存储空间自动回收机制
items: 使用docker-ls查看容器仓库的镜像、标签等信息
items: 镜像kubesphere镜像到本地harbor
items: 镜像dockerhub脚本 - 更换docker.io token方式
items: 镜像dockerhub脚本 - 更换源的方式
items: 使用registry:2部署容器镜像加速站点

运维用容器镜像仓库
type: item
date: 2024-07-17T05:08:00+00:00
description: 前一阵子搭建了一个harbor镜像仓库，并且写了搬砖脚本，只是提供一些运维用的到的镜像仓库。如果大家有需要，可以试试。如果需要一些运维工作相关的镜像，可以留言，我会及时同步。镜像仓库地址：harbor.op123.ren:44301 修改docker配置：cat <<'EOF'>/etc/docker/daemon.json { "debug": false, "experimental": true, "registry-mirrors": [ "https://harbor.op123.ren:44301" ] } EOF systemctl daemon-reload systemctl restart docker 如此一来，既方便大家使用，也可以丰富仓库内容。让运维工作更顺畅自然。脚本内容如下：#!/usr/bin/python3 # filename: dockerpullandpush.py # 部署在hk小鸡里 import docker import argparse import subprocess import json def pull_and_push_image(image_name, docker_username, docker_password, harbor_url, harbor_username, harbor_password): # 从 image_name 中提取 harbor_project if '/' in image_name: harbor_project = image_name.split('/')[0] image_name_without_project = '/'.join(image_name.split('/')[1:]) else: harbor_project = 'library' image_name_without_project = image_name # 创建 Docker 客户端 client = docker.from_env() # 登录 Docker Hub client.login(username=docker_username, password=docker_password) # 拉取镜像 print(f"Pulling image {image_name} from Docker Hub...") image = client.images.pull(image_name) # 标记镜像 harbor_image_name = f"{harbor_url}/{harbor_project}/{image_name_without_project}" image.tag(harbor_image_name) # 登录 Harbor client.login(username=harbor_username, password=harbor_password, registry=harbor_url) # 检查项目是否存在 project_exists_cmd = f"curl -s -u {harbor_username}:{harbor_password} -X GET https://{harbor_url}/api/v2.0/projects?name={harbor_project}" project_exists_output = subprocess.check_output(project_exists_cmd, shell=True) project_exists_data = json.loads(project_exists_output) if len(project_exists_data) == 0: # 如果项目不存在,则创建项目 print(f"Project {harbor_project} does not exist in Harbor. Creating...") create_project_cmd = f"curl -s -u {harbor_username}:{harbor_password} -X POST -H \"Content-Type: application/json\" -d '{{\"project_name\": \"{harbor_project}\", \"public\": true}}' https://{harbor_url}/api/v2.0/projects" subprocess.check_output(create_project_cmd, shell=True) print(f"Project {harbor_project} creat…

harbor - 简单做了一个私域容器镜像仓库
type: item
date: 2024-07-01T06:53:00+00:00
description: 为了方便日常的工作等使用，搭建一个私域的容器镜像仓库。搭建的方法也简单。直接下载harbor离线安装包，解压，复制harbor.yml.sample文件，修改其中的域名、端口、证书等信息，然后运行install.sh，等待服务启动。写个脚本dockerpullandpush.py，通过运行+镜像名称，实现从dockerhub下载并上传到私钥harbor镜像仓库。脚本内容如下：#!/usr/bin/python3.7 import docker import argparse import subprocess import json def pull_and_push_image(image_name, docker_username, docker_password, harbor_url, harbor_username, harbor_password): # 从 image_name 中提取 harbor_project if '/' in image_name: harbor_project = image_name.split('/')[0] else: harbor_project = 'library' # 创建 Docker 客户端 client = docker.from_env() # 登录 Docker Hub client.login(username=docker_username, password=docker_password) # 拉取镜像 print(f"Pulling image {image_name} from Docker Hub...") image = client.images.pull(image_name) # 标记镜像 harbor_image_name = f"{harbor_url}/{harbor_project}/{image_name}" image.tag(harbor_image_name) # 登录 Harbor client.login(username=harbor_username, password=harbor_password, registry=harbor_url) # 检查项目是否存在 project_exists_cmd = f"curl -s -u {harbor_username}:{harbor_password} -X GET https://{harbor_url}/api/v2.0/projects?name={harbor_project}" project_exists_output = subprocess.check_output(project_exists_cmd, shell=True) project_exists_data = json.loads(project_exists_output) if len(project_exists_data) == 0: # 如果项目不存在,则创建项目 print(f"Project {harbor_project} does not exist in Harbor. Creating...") create_project_cmd = f"curl -s -u {harbor_username}:{harbor_password} -X POST -H \"Content-Type: application/json\" -d '{{\"project_name\": \"{harbor_project}\", \"public\": true}}' https://{harbor_url}/api/v2.0/projects" subprocess.check_output(create_project_cmd, shell=True) print(f"Project {harbor_project} created successfully.") # 推送镜像到 Harbor print(f"Pushing image {harbor_image_name} to Harbor...") client.images.push(harbor_image_name) print("Image pulled and pushed successfully.") # 清理下载的容器镜像 print(f"Cleaning up downloaded image {image_name}...") client.images.remove(image_name) client.images.remove(harbor_image_name) print("Cleanup …

arm aarch x86_64这些属于啥，关系是啥，还有其他的架构么？
type: item
date: 2023-11-15T08:04:00+00:00
description: ARM, AArch64, 和 x86_64 都是计算机处理器的架构。它们定义了处理器如何处理数据，如何执行指令，以及如何与其他硬件交互。ARM: ARM 是 Advanced RISC Machine 的缩写，它是一种采用 RISC（Reduced Instruction Set Computing，精简指令集计算）设计的处理器架构。ARM 架构的处理器通常在移动设备（如智能手机和平板电脑）和嵌入式系统（如路由器和物联网设备）中使用，因为它们通常具有较低的功耗。AArch64: AArch64 是 ARMv8 架构中的 64 位模式，它是 ARM 架构的扩展。AArch64 支持更多的寄存器，以及更大的内存寻址空间，这使得它在需要处理大量数据的应用（如图形处理和机器学习）中具有优势。x86_64: x86_64 也被称为 AMD64 或 x64，它是 x86 架构的 64 位版本。x86 架构最初由 Intel 开发，最早的版本是 16 位的，后来扩展到 32 位（即 IA-32 或 x86），最后扩展到 64 位（即 x86_64）。x86_64 架构的处理器在个人电脑和服务器中广泛使用。除了这些，还有其他的处理器架构，例如：MIPS: MIPS 是 Microprocessor without Interlocked Pipelined Stages 的缩写，它也是一种采用 RISC 设计的处理器架构。MIPS 架构的处理器在很多嵌入式系统中使用。PowerPC: PowerPC 是一种由 Apple、IBM 和 Motorola 共同开发的 RISC 处理器架构。虽然现在在个人电脑中已经不常见，但它仍在一些嵌入式系统和高性能计算系统中使用。SPARC: SPARC 是 Scalable Processor Architecture 的缩写，它是一种由 Sun Microsystems 开发的 RISC 处理器架构。SPARC 架构的处理器主要在一些高端的服务器和工作站中使用。每种处理器架构都有其自己的优点和特点，选择哪种架构取决于特定的应用需求，如性能、功耗、成本等因素。现在有一个容器镜像，架构是amd64，那么对于os有没有要求？是否可以直接linux和windows里面使用呢？容器镜像的架构（例如 amd64）是指镜像是为哪种处理器架构构建的。这个架构需要与运行容器的主机的处理器架构相匹配。也就是说，如果你有一个为 amd64 架构构建的容器镜像，那么你需要在一个 amd64 架构的主机上运行它。至于操作系统，容器技术最初是在 Linux 上开发的，因为它依赖于 Linux 内核的一些特性，如 cgroups 和 namespaces。这意味着，如果你有一个为 Linux 构建的容器镜像，那么你需要在 Linux 主机上运行它。然而，随着容器技术的发展，Microsoft 在 Windows 10 和 Windows Server 2016 中引入了对 Windows 容器的支持。这意味着你可以在 Windows 主机上运行为 Windows 构建的容器镜像。但是，你不能在 Windows 主机上运行为 Linux 构建的容器镜像，反之亦然。如果你需要在 Windows 主机上运行 Linux 容器，你可以使用 Docker Desktop for Windows，它使用的是一个名为 MobyLinuxVM 的轻量级虚拟机来运行 Linux 容器。或者，你也可以使用 Windows Subsystem for Linux (WSL)，它允许你在 Windows 中运行 Linux 环境。总的来说，容器镜像的架构和操作系统都需要与运行容器的主机匹配。如果不匹配，你可能需要使用一些工具或技术来解决这个问题。WSL是一个linux虚拟机么？Windows Subsystem for Linux (WSL) 不是传统意义上的 Linux 虚拟机，它更像是一个兼容层，使得你可以在 Windows 系统上直接运行 Linux 二进制可执行文件。WSL 1 和 WSL 2 的工作原理有一些不同：WSL 1：WSL 1 是一个兼容层，它翻译 Linux 系统调用为 Windows 系统调用，然后让 Linux 二进制可执行文件在 Windows 内核上运行。这种方法的优点是性能开销小，但缺点是一些复杂的 Linux 系统调用（例如，那些涉及到文件系统或网络堆栈的）可能无法完全或正确地翻译。WSL 2：WSL 2 是一个轻量级虚拟机，它运行一个完整的 Linux 内核，并在其上运行 Linux 二进制可执行文件。这种方法的优点是兼容性更好（因为它使用的是真正的 Linux 内核），但缺点是有一些额外的性能开销（虽然 Microsof…

镜像dockerhub到本地harbor-20231114
type: item
date: 2023-11-14T09:10:00+00:00
description: 为了更好的使用容器镜像，以及常见的收集习惯，尝试将dockerhub中的镜像，或者主要镜像同步到本地镜像仓库harbor。同步的方法主要是设定上游服务器docker.io，dockerproxy.com。缺点，两个都有限制。近期发现，AWS有ECS服务，地址public.ecr.aws部署下游harbor服务器，本地可以访问。设定一批待同步projects进入bash shell脚本的数组。通过docker-ls访问上游服务器并拿到tag列表，也访问下游harbor服务器并达到已有的tag列表，如果没有就进行pull，然后push操作。主要脚本内容如下：#!/bin/bash set -x harbor1="public.ecr.aws" harbor2="harbor.sddts.cn" docker login -u admin -p "xxx" harbor.sddts.cn projects=() projects+=("library/mysql" "library/alpine" "library/centos" "library/debian" "library/ubuntu" "library/python" "library/caddy" "library/redis" "library/nginx" "library/busybox" "library/postgres" "library/node" "library/memcached" "library/httpd" "library/mongo" "library/rabbitmq" "library/mariadb" "library/docker" "library/hello-world" "library/openjdk" "library/golang" "library/registry" "library/wordpress" "library/php" "library/consul" "library/influxdb" "library/haproxy" "library/tomcat" "library/maven" "library/bash" "library/ghost" "library/kong" "library/zookeeper" "jenkins/jenkins" "library/mediawiki" "library/django" "library/gcc") #projects+=("chaitin/safeline-tengine" "chaitin/safeline-mgt-api" "chaitin/safeline-detector" "chaitin/safeline-mario") #projects+=("jumpserver/lion" "jumpserver/koko" "jumpserver/guacamole" "jumpserver/core" "jumpserver/jms_all" "jumpserver/web" "jumpserver/redis" "jumpserver/nginx" "jumpserver/core" "jumpserver/mysql" "jumpserver/magnus" "jumpserver/mariadb" "jumpserver/luna" "jumpserver/guacamole" "jumpserver/lina" "jumpserver/chen" "jumpserver/kael" "jumpserver/guacd" "jumpserver/python" "jumpserver/jms_web") #projects+=("flashcatcloud/categraf" "flashcatcloud/nightingale" "flashcatcloud/ibex" "flashcatcloud/chatshot" "flashcatcloud/mock" "flashcatcloud/toolbox" "flashcatcloud/rocketmq-exporter" "flashcatcloud/public" "flashcatcloud/chrome" "flashcatcloud/scvpn" "flashcatcloud/scvpn" "flashcatcloud/wine" "flashcatcloud/kube-state-metric" "flashcatcloud/corplink" ) #projects+=("joyqi/typecho") #projects+=("joyqi/typecho" ) …

清理并部署harbor存储空间自动回收机制
type: item
date: 2023-11-14T07:35:00+00:00
description: 周一，研发同事发来一张截图，大概意思是发布失败。分析jenkins日志，发现是镜像空间存储满了，导致无法将jenkins生成的容器镜像上传到harbor服务。删除镜像点击 “项目” -> “某个项目名称” -> “进入仓库” -> “选择要删除的制品”然后在操作栏下拉中选择删除。注意，这个时候并没有真正释放空间。点击“系统管理”->“垃圾清理”->“立即清理垃圾”。增加回收机制点击 “项目” -> “project” -> “Policy” -> “Add Rule”然后选择保留最新的10份tag然后点击DRY RUN，看看有哪些镜像会被删除。点击RUN NOW可以执行回收工作。增加定时执行后记为啥监控没有告警呢？查看了下夜莺监控，原来是没有开启主机监控项，配置了相关cpu、内存、磁盘的选项后，就可以收到告警了。

使用docker-ls查看容器仓库的镜像、标签等信息
type: item
date: 2023-11-02T07:42:00+00:00
description: docker-ls是啥？下载安装docker-lsif [ ! -e /bin/docker-ls ]; then wget https://vip.123pan.cn/1815238395/download/docker-ls/docker-ls-linux-amd64.zip -O /tmp/docker-ls-linux-amd64.zip unzip -d /bin/ /tmp/docker-ls-linux-amd64.zip fi 使用列出有多少个仓库docker-ls repositories -r https://docker.io docker-ls repositories -r https://harbor.sddts.cn --basic-auth -u admin -p xxx 列出某个仓库的标签docker-ls tags -r https://docker.io library/mysql

镜像kubesphere镜像到本地harbor
type: item
date: 2023-11-02T06:27:00+00:00
description: kubesphere是国内优秀的基于kubernetes开发的适合国内使用的k8s集群软件，部署简单，操作简单。为了进一步提高本地搭建测试效率，尝试将其安装部署中的各种容器放置到本地harbor仓库中。拿到kubesphere需要的容器镜像名称使用该链接，在Linuux上安装的离线安装部分，包含所有容器镜像内容[collapse status="false" title="容器镜像内容"] images: - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-apiserver:v1.23.15 - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-controller-manager:v1.23.15 - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-proxy:v1.23.15 - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-scheduler:v1.23.15 - registry.cn-beijing.aliyuncs.com/kubesphereio/pause:3.6 - registry.cn-beijing.aliyuncs.com/kubesphereio/coredns:1.8.6 - registry.cn-beijing.aliyuncs.com/kubesphereio/cni:v3.23.2 - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-controllers:v3.23.2 - registry.cn-beijing.aliyuncs.com/kubesphereio/node:v3.23.2 - registry.cn-beijing.aliyuncs.com/kubesphereio/pod2daemon-flexvol:v3.23.2 - registry.cn-beijing.aliyuncs.com/kubesphereio/typha:v3.23.2 - registry.cn-beijing.aliyuncs.com/kubesphereio/flannel:v0.12.0 - registry.cn-beijing.aliyuncs.com/kubesphereio/provisioner-localpv:3.3.0 - registry.cn-beijing.aliyuncs.com/kubesphereio/linux-utils:3.3.0 - registry.cn-beijing.aliyuncs.com/kubesphereio/haproxy:2.3 - registry.cn-beijing.aliyuncs.com/kubesphereio/nfs-subdir-external-provisioner:v4.0.2 - registry.cn-beijing.aliyuncs.com/kubesphereio/k8s-dns-node-cache:1.15.12 - registry.cn-beijing.aliyuncs.com/kubesphereio/ks-installer:v3.4.0 - registry.cn-beijing.aliyuncs.com/kubesphereio/ks-apiserver:v3.4.0 - registry.cn-beijing.aliyuncs.com/kubesphereio/ks-console:v3.4.0 - registry.cn-beijing.aliyuncs.com/kubesphereio/ks-controller-manager:v3.4.0 - registry.cn-beijing.aliyuncs.com/kubesphereio/kubectl:v1.22.0 - registry.cn-beijing.aliyuncs.com/kubesphereio/kubectl:v1.21.0 - registry.cn-beijing.aliyuncs.com/kubesphereio/kubectl:v1.20.0 - registry.cn-beijing.aliyuncs.com/kubesphereio/kubefed:v0.8.1 - registry.cn-beijing.aliyuncs.com/kubesphereio/tower:v0.2.1 - registry.cn-beijing.aliyuncs…

镜像dockerhub脚本 - 更换docker.io token方式
type: item
date: 2023-11-01T07:25:00+00:00
description: #!/bin/bash harbor2="harbor.sddts.cn" dockerhubTokens=("zzlyzq:111" "zzlyzq1:111") docker login -u zzlyzq -p "111" docker.io docker login -u admin -p "111" harbor.sddts.cn #projects=("library/mysql" "library/alpine") projects=("library/mysql" "library/alpine" "library/centos" "library/debian" "library/ubuntu" "library/python" "library/redis" "library/nginx" "library/busybox" "library/postgres" "library/node" "library/memcached" "library/httpd" "library/mongo" "library/rabbitmq" "library/mariadb" "library/docker" "library/hello-world" "library/openjdk" "library/golang" "library/registry" "library/wordpress" "library/php" "library/consul" "library/influxdb" "library/haproxy" "library/tomcat" "library/maven" "library/bash" "library/ghost" "library/kong" "library/zookeeper" "jenkins/jenkins" "library/mediawiki" "library/django" "library/gcc" "chaitin/safeline-tengine" "chaitin/safeline-mgt-api" "chaitin/safeline-detector" "chaitin/safeline-mario" "jumpserver/lion" "jumpserver/koko" "jumpserver/guacamole" "jumpserver/core" "jumpserver/jms_all" "jumpserver/web" "jumpserver/redis" "jumpserver/nginx" "jumpserver/core" "jumpserver/mysql" "jumpserver/magnus" "jumpserver/mariadb" "jumpserver/luna" "jumpserver/guacamole" "jumpserver/lina" "jumpserver/chen" "jumpserver/kael" "jumpserver/guacd" "jumpserver/python" "jumpserver/jms_web") if [ ! -e /bin/docker-ls ]; then wget https://vip.123pan.cn/1815238395/download/docker-ls/docker-ls-linux-amd64.zip -O /tmp/docker-ls-linux-amd64.zip unzip -d /bin/ /tmp/docker-ls-linux-amd64.zip fi for project in "${projects[@]}"; do echo "开始新的project" echo $project tagList=$(docker-ls tags $project 2>/dev/null | grep '\- ' | tr -d '"' | awk '{print $2}') echo "tagList is $tagList" tagList2=$(docker-ls tags -r https://$harbor2 -u admin -p 111 $project 2>/dev/null | grep '\- ' | tr -d '"' | awk '{print $2}') echo "tagList2 is $tagList2" for tag in $tagList; do if [ "$tag" == "latest" ]; then echo "发现latest TAG标签" else if [ $(echo $tagList2 | grep $…

镜像dockerhub脚本 - 更换源的方式
type: item
date: 2023-11-01T05:11:00+00:00
description: #!/bin/bash harbor2="harbor.sddts.cn" harbor1List=("docker.io" "dockerproxy.com" "docker.nju.edu.cn") docker login -u zzlyzq -p "111" docker.io docker login -u admin -p "123" harbor.sddts.cn #projects=("library/mysql" "library/alpine") projects=("library/mysql" "library/alpine" "library/centos" "library/debian" "library/ubuntu" "library/python" "library/redis" "library/nginx" "library/busybox" "library/postgres" "library/node" "library/memcached" "library/httpd" "library/mongo" "library/rabbitmq" "library/mariadb" "library/docker" "library/hello-world" "library/openjdk" "library/golang" "library/registry" "library/wordpress" "library/php" "library/consul" "library/influxdb" "library/haproxy" "library/tomcat" "library/maven" "library/bash" "library/ghost" "library/kong" "library/zookeeper" "jenkins/jenkins" "library/mediawiki" "library/django" "library/gcc" "chaitin/safeline-tengine" "chaitin/safeline-mgt-api" "chaitin/safeline-detector" "chaitin/safeline-mario" "jumpserver/lion" "jumpserver/koko" "jumpserver/guacamole" "jumpserver/core" "jumpserver/jms_all" "jumpserver/web" "jumpserver/redis" "jumpserver/nginx" "jumpserver/core" "jumpserver/mysql" "jumpserver/magnus" "jumpserver/mariadb" "jumpserver/luna" "jumpserver/guacamole" "jumpserver/lina" "jumpserver/chen" "jumpserver/kael" "jumpserver/guacd" "jumpserver/python" "jumpserver/jms_web") if [ ! -e /bin/docker-ls ]; then wget https://vip.123pan.cn/1815238395/download/docker-ls/docker-ls-linux-amd64.zip -O /tmp/docker-ls-linux-amd64.zip unzip -d /bin/ /tmp/docker-ls-linux-amd64.zip fi for project in "${projects[@]}"; do echo $project tagList=$(docker-ls tags -r https://docker.io -u zzlyzq -p "123" $project 2>/dev/null | grep '\- ' | tr -d '"' | awk '{print $2}') tagList2=$(docker-ls tags -r https://$harbor2 -u admin -p 123 $project 2>/dev/null | grep '\- ' | tr -d '"' | awk '{print $2}') for tag in $tagList; do if [ "$tag" = "latest" ]; then continue else #ifExistsIn=`echo $tagList2 | grep $tag | wc -l` if [ $(echo…

使用registry:2部署容器镜像加速站点
type: item
date: 2023-10-30T07:35:00+00:00
description: docker-compose文件version: '3' services: registry: image: registry:2 ports: - 172.31.20.2:5000:5000 volumes: - ./data:/var/lib/registry networks: mynetwork: ipv4_address: 172.31.20.2 networks: mynetwork: ipam: config: - subnet: 172.31.20.0/24 客户端文件{ "registry-mirrors": ["http://localhost:5000"], "insecure-registries": ["localhost:5000"] }