tekton-pipeline-without-pipeline-resources.html (57981B)
1 <!DOCTYPE html> 2 <html lang="en"> 3 <head> 4 <!-- Sep 03, 2024 --> 5 <meta charset="utf-8" /> 6 <meta name="viewport" content="width=device-width, initial-scale=1" /> 7 <title>Tekton Pipeline : another world without PipelineResources</title> 8 <meta name="author" content="Vincent Demeester" /> 9 <meta name="generator" content="Org Mode" /> 10 <link rel='icon' type='image/x-icon' href='/images/favicon.ico'/> 11 <meta name='viewport' content='width=device-width, initial-scale=1'> 12 <link rel='stylesheet' href='/css/new.css' type='text/css'/> 13 <link rel='stylesheet' href='/css/syntax.css' type='text/css'/> 14 <link href='/index.xml' rel='alternate' type='application/rss+xml' title='Vincent Demeester' /> 15 </head> 16 <body> 17 <main id="content" class="content"> 18 <header> 19 <h1 class="title">Tekton Pipeline : another world without PipelineResources</h1> 20 </header><p> 21 This document is a refreshed version of <a href="https://docs.google.com/document/d/1u6qO7CPtDnTOZMYFQ5ARysSOy8lfFeVw83C_5BxAKsw/edit#heading=h.yc5nzf2ze0dr">A world without PipelineResource</a> (only accessible 22 if you are a member of the tekton community) as of <code>tekton/pipeline</code> <code>v0.17.x</code>. 23 </p> 24 <section id="outline-container-Goal%28s%29" class="outline-2"> 25 <h2 id="Goal%28s%29"><span class="todo TODO">TODO</span> Goal(s)</h2> 26 <div class="outline-text-2" id="text-Goal%28s%29"> 27 </div> 28 <div id="outline-container-PipelineResources%20%22problems%22" class="outline-3"> 29 <h3 id="PipelineResources%20%22problems%22">PipelineResources “problems”</h3> 30 <div class="outline-text-3" id="text-PipelineResources%20%22problems%22"> 31 </div> 32 <div id="outline-container-PipelineResources%20extra%28s%29" class="outline-4"> 33 <h4 id="PipelineResources%20extra%28s%29">PipelineResources extra(s)</h4> 34 <div class="outline-text-4" id="text-PipelineResources%20extra%28s%29"> 35 <p> 36 Related issue is <a href="https://github.com/tektoncd/pipeline/issues/3518">#3518</a>, but there might be others. It is not currently possible to pass 37 extra certificates to a <code>PipelineResource</code> generated container, making, for example a 38 self-signed <code>https</code> git clone from using <code>PipelineResource</code> impossible. This may apply to 39 additional “extra” that we would want to apply to <code>PipelineResource</code> that we can apply to 40 <code>Task</code> (additional volumes, …). 41 </p> 42 </div> 43 </div> 44 </div> 45 </section> 46 <section id="outline-container-Examples" class="outline-2"> 47 <h2 id="Examples"><span class="todo TODO">TODO</span> Examples</h2> 48 <div class="outline-text-2" id="text-Examples"> 49 <div class='drawer logbook'> 50 <h6>Logbook</h6> 51 nil</div> 52 53 <p> 54 The examples in the document are based on the “User stories” list of <a href="https://docs.google.com/document/d/1h9n0Lod0OiJ_sP2HK8Ms7N04aee5LW8xfz5yMGCFMIs/edit?ts=5f96a3e8#">Tekton Pipeline v1 55 API</a> document. 56 </p> 57 58 <p> 59 We are going to use the <a href="https://github.com/tektoncd/catalog">catalog</a> task as much as we can. We are also going to use tekton 60 bundle, that will be available starting from <code>v0.18.0</code>. 61 </p> 62 </div> 63 <div id="outline-container-Prerequisite" class="outline-3"> 64 <h3 id="Prerequisite"><span class="done DONE">DONE</span> Prerequisite</h3> 65 <div class="outline-text-3" id="text-Prerequisite"> 66 <div class='drawer logbook'> 67 <h6>Logbook</h6> 68 <ul class="org-ul"> 69 <li>State “DONE” from “TODO” <span class="timestamp-wrapper"><span class="timestamp">[2020-11-02 Mon 15:36]</span></span></li> 70 </ul> 71 </div> 72 73 <p> 74 Let’s bundle some base tasks into a tekton bundle to “ease” of use. 75 </p> 76 77 <ul class="org-ul"> 78 <li><p> 79 <code>git</code> 80 </p> 81 82 <div class="org-src-container"> 83 <pre class="src src-bash">tkn-oci push docker.io/vdemeester/tekton-base-git:v0.1 task/git-clone/0.2/git-clone.yaml task/git-cli/0.1/git-cli.yaml task/git-rebase/0.1/git-rebase.yaml 84 </pre> 85 </div></li> 86 </ul> 87 88 89 <p> 90 Let’s also create a <i>generic</i> PVC for getting source code, … 91 </p> 92 93 <div class="org-src-container"> 94 <pre class="src src-yaml">apiVersion: v1 95 kind: PersistentVolumeClaim 96 metadata: 97 name: testpvc 98 spec: 99 accessModes: [ ReadWriteMany ] 100 storageClassName: standard 101 resources: 102 requests: 103 storage: 1Gi 104 </pre> 105 </div> 106 </div> 107 </div> 108 <div id="outline-container-Standard%20Go%20Pipeline" class="outline-3"> 109 <h3 id="Standard%20Go%20Pipeline"><span class="done DONE">DONE</span> Standard Go Pipeline</h3> 110 <div class="outline-text-3" id="text-Standard%20Go%20Pipeline"> 111 <div class='drawer logbook'> 112 <h6>Logbook</h6> 113 <ul class="org-ul"> 114 <li>State “DONE” from “TODO” <span class="timestamp-wrapper"><span class="timestamp">[2020-11-02 Mon 15:34]</span></span></li> 115 </ul> 116 </div> 117 118 <p> 119 A simple go pipeline is doing the following: 120 </p> 121 <ul class="org-ul"> 122 <li><b>linting</b> using <code>golangci-lint</code> - <a href="https://raw.githubusercontent.com/tektoncd/catalog/master/task/golangci-lint/0.1/golangci-lint.yaml">golangci-lint</a></li> 123 <li><b>build</b> using <code>go build</code> - <a href="https://raw.githubusercontent.com/tektoncd/catalog/master/task/golang-build/0.1/golang-build.yaml">golang-build</a></li> 124 <li><b>testing</b> using <code>go test</code> - <a href="https://raw.githubusercontent.com/tektoncd/catalog/master/task/golang-test/0.1/golang-test.yaml">golang-test</a></li> 125 </ul> 126 127 <div class="org-src-container"> 128 <pre class="src src-bash">tkn-oci push docker.io/vdemeester/tekton-golang:v0.1 task/golangci-lint/0.1/golangci-lint.yaml task/golang-build/0.1/golang-build.yaml task/golang-test/0.1/golang-test.yaml 129 </pre> 130 </div> 131 132 <div class="org-src-container"> 133 <pre class="src src-yaml">--- 134 apiVersion: tekton.dev/v1beta1 135 kind: Pipeline 136 metadata: 137 name: std-golang 138 spec: 139 params: 140 - name: package 141 #- name: url 142 # default: https://$(params.package) # that doesn't work 143 - name: revision 144 default: "" 145 workspaces: 146 - name: ws 147 tasks: 148 - name: fetch-repository 149 taskRef: 150 name: git-clone 151 bundle: docker.io/vdemeester/tekton-base-git:v0.1 152 workspaces: 153 - name: output 154 workspace: ws 155 params: 156 - name: url 157 value: https://$(params.package) 158 - name: build 159 taskRef: 160 name: golang-build 161 bundle: docker.io/vdemeester/tekton-golang:v0.1 162 runAfter: [ fetch-repository ] 163 params: 164 - name: package 165 value: $(params.package) 166 workspaces: 167 - name: source 168 workspace: ws 169 - name: lint 170 taskRef: 171 name: golangci-lint 172 bundle: docker.io/vdemeester/tekton-golang:v0.1 173 runAfter: [ fetch-repository ] 174 params: 175 - name: package 176 value: $(params.package) 177 workspaces: 178 - name: source 179 workspace: ws 180 - name: test 181 taskRef: 182 name: golang-test 183 bundle: docker.io/vdemeester/tekton-golang:v0.1 184 runAfter: [ build, lint ] 185 params: 186 - name: package 187 value: $(params.package) 188 workspaces: 189 - name: source 190 workspace: ws 191 </pre> 192 </div> 193 194 <div class="org-src-container"> 195 <pre class="src src-yaml">apiVersion: tekton.dev/v1beta1 196 kind: PipelineRun 197 metadata: 198 generateName: run-std-go- 199 spec: 200 pipelineRef: 201 name: std-golang 202 params: 203 - name: package 204 value: github.com/tektoncd/pipeline 205 workspaces: 206 - name: ws 207 volumeClaimTemplate: 208 spec: 209 accessModes: 210 - ReadWriteMany 211 resources: 212 requests: 213 storage: 1Gi 214 </pre> 215 </div> 216 217 <p> 218 Note: 219 </p> 220 <ul class="org-ul"> 221 <li><code>bundle</code> is duplicated a lot (default bundle would reduce verbosity).</li> 222 <li><code>params</code> and <code>workspaces</code> are duplicated in there. 223 <i>Maybe we could be able to specify workspace to be available for all tasks</i></li> 224 </ul> 225 </div> 226 </div> 227 <div id="outline-container-Standard%20Java%20Pipeline%28s%29" class="outline-3"> 228 <h3 id="Standard%20Java%20Pipeline%28s%29"><span class="done DONE">DONE</span> Standard Java Pipeline(s)</h3> 229 <div class="outline-text-3" id="text-Standard%20Java%20Pipeline%28s%29"> 230 <div class='drawer logbook'> 231 <h6>Logbook</h6> 232 <ul class="org-ul"> 233 <li>State “DONE” from “TODO” <span class="timestamp-wrapper"><span class="timestamp">[2020-11-02 Mon 17:23]</span></span></li> 234 </ul> 235 </div> 236 237 <div class="org-src-container"> 238 <pre class="src src-bash">tkn-oci push docker.io/vdemeester/tekton-java:v0.1 task/maven/0.2/maven.yaml task/jib-gradle/0.1/jib-gradle.yaml task/jib-maven/0.1/jib-maven.yaml 239 </pre> 240 </div> 241 </div> 242 <div id="outline-container-Standard%20Java%20Pipeline%28s%29--Prerequisite" class="outline-4"> 243 <h4 id="Standard%20Java%20Pipeline%28s%29--Prerequisite"><span class="done DONE">DONE</span> Prerequisite</h4> 244 <div class="outline-text-4" id="text-Standard%20Java%20Pipeline%28s%29--Prerequisite"> 245 <div class='drawer logbook'> 246 <h6>Logbook</h6> 247 <ul class="org-ul"> 248 <li>State “DONE” from “TODO” <span class="timestamp-wrapper"><span class="timestamp">[2020-11-02 Mon 16:27]</span></span></li> 249 </ul> 250 </div> 251 252 <p> 253 Let’s have a <code>nexus</code> server running… 254 </p> 255 256 <div class="org-src-container"> 257 <pre class="src src-yaml">--- 258 apiVersion: apps/v1 259 kind: Deployment 260 metadata: 261 labels: 262 app: nexus 263 app.kubernetes.io/instance: nexus 264 app.kubernetes.io/name: nexus 265 app.kubernetes.io/part-of: nexus 266 name: nexus 267 spec: 268 replicas: 1 269 selector: 270 matchLabels: 271 app: nexus 272 template: 273 metadata: 274 labels: 275 app: nexus 276 spec: 277 containers: 278 - name: nexus 279 image: docker.io/sonatype/nexus3:3.16.2 280 env: 281 - name: CONTEXT_PATH 282 value: / 283 imagePullPolicy: IfNotPresent 284 ports: 285 - containerPort: 8081 286 protocol: TCP 287 livenessProbe: 288 exec: 289 command: 290 - echo 291 - ok 292 failureThreshold: 3 293 initialDelaySeconds: 30 294 periodSeconds: 10 295 successThreshold: 1 296 timeoutSeconds: 1 297 readinessProbe: 298 failureThreshold: 3 299 httpGet: 300 path: / 301 port: 8081 302 scheme: HTTP 303 initialDelaySeconds: 30 304 periodSeconds: 10 305 successThreshold: 1 306 timeoutSeconds: 1 307 resources: 308 limits: 309 memory: 4Gi 310 cpu: 2 311 requests: 312 memory: 512Mi 313 cpu: 200m 314 terminationMessagePath: /dev/termination-log 315 volumeMounts: 316 - mountPath: /nexus-data 317 name: nexus-data 318 volumes: 319 - name: nexus-data 320 persistentVolumeClaim: 321 claimName: nexus-pv 322 --- 323 apiVersion: v1 324 kind: Service 325 metadata: 326 labels: 327 app: nexus 328 name: nexus 329 spec: 330 ports: 331 - name: 8081-tcp 332 port: 8081 333 protocol: TCP 334 targetPort: 8081 335 selector: 336 app: nexus 337 sessionAffinity: None 338 type: ClusterIP 339 # --- 340 # apiVersion: v1 341 # kind: Route 342 # metadata: 343 # labels: 344 # app: nexus 345 # name: nexus 346 # spec: 347 # port: 348 # targetPort: 8081-tcp 349 # to: 350 # kind: Service 351 # name: nexus 352 # weight: 100 353 --- 354 apiVersion: v1 355 kind: PersistentVolumeClaim 356 metadata: 357 labels: 358 app: nexus 359 name: nexus-pv 360 spec: 361 accessModes: 362 - ReadWriteOnce 363 resources: 364 requests: 365 storage: 5Gi 366 </pre> 367 </div> 368 369 <p> 370 … a maven-repo PVC 371 </p> 372 373 <div class="org-src-container"> 374 <pre class="src src-yaml">apiVersion: v1 375 kind: PersistentVolumeClaim 376 metadata: 377 name: maven-repo-pvc 378 spec: 379 resources: 380 requests: 381 storage: 5Gi 382 volumeMode: Filesystem 383 accessModes: 384 - ReadWriteMany 385 </pre> 386 </div> 387 388 <p> 389 and a maven settings configmap 390 </p> 391 392 <div class="org-src-container"> 393 <pre class="src src-yaml">apiVersion: v1 394 kind: ConfigMap 395 metadata: 396 name: custom-maven-settings 397 data: 398 settings.xml: | 399 <?xml version="1.0" encoding="UTF-8"?> 400 <settings> 401 <servers> 402 <server> 403 <id>nexus</id> 404 <username>admin</username> 405 <password>admin123</password> 406 </server> 407 </servers> 408 <mirrors> 409 <mirror> 410 <id>nexus</id> 411 <name>nexus</name> 412 <url>http://nexus:8081/repository/maven-public/</url> 413 <mirrorOf>*</mirrorOf> 414 </mirror> 415 </mirrors> 416 </settings> 417 </pre> 418 </div> 419 </div> 420 </div> 421 <div id="outline-container-Maven" class="outline-4"> 422 <h4 id="Maven"><span class="done DONE">DONE</span> Maven</h4> 423 <div class="outline-text-4" id="text-Maven"> 424 <div class='drawer logbook'> 425 <h6>Logbook</h6> 426 <ul class="org-ul"> 427 <li>State “DONE” from “TODO” <span class="timestamp-wrapper"><span class="timestamp">[2020-11-02 Mon 16:40]</span></span></li> 428 </ul> 429 </div> 430 431 <p> 432 A simple <code>maven</code> project pipeline that build, run test, packages and publish artifacts 433 (jars) to a maven repository. <i>Note: it uses a maven cache (<code>.m2</code>)</i>. 434 </p> 435 436 <p> 437 The pipeline… 438 </p> 439 440 <div class="org-src-container"> 441 <pre class="src src-yaml">apiVersion: tekton.dev/v1beta1 442 kind: Pipeline 443 metadata: 444 name: std-maven 445 spec: 446 params: 447 - name: url 448 - name: revision 449 default: "" 450 workspaces: 451 - name: ws 452 - name: local-maven-repo 453 - name: maven-settings 454 optional: true 455 tasks: 456 - name: fetch-repository 457 taskRef: 458 name: git-clone 459 bundle: docker.io/vdemeester/tekton-base-git:v0.1 460 workspaces: 461 - name: output 462 workspace: ws 463 params: 464 - name: url 465 value: $(params.url) 466 - name: unit-tests 467 taskRef: 468 bundle: docker.io/vdemeester/tekton-java:v0.1 469 name: maven 470 runAfter: 471 - fetch-repository 472 workspaces: 473 - name: source 474 workspace: ws 475 - name: maven-repo 476 workspace: local-maven-repo 477 - name: maven-settings 478 workspace: maven-settings 479 params: 480 - name: GOALS 481 value: ["package"] 482 - name: release-app 483 taskRef: 484 bundle: docker.io/vdemeester/tekton-java:v0.1 485 name: maven 486 runAfter: 487 - unit-tests 488 workspaces: 489 - name: source 490 workspace: ws 491 - name: maven-repo 492 workspace: local-maven-repo 493 - name: maven-settings 494 workspace: maven-settings 495 params: 496 - name: GOALS 497 value: 498 - deploy 499 - -DskipTests=true 500 - -DaltDeploymentRepository=nexus::default::http://nexus:8081/repository/maven-releases/ 501 - -DaltSnapshotDeploymentRepository=nexus::default::http://nexus:8081/repository/maven-snapshots/ 502 </pre> 503 </div> 504 505 <p> 506 … and the pipeline run 507 </p> 508 509 <div class="org-src-container"> 510 <pre class="src src-yaml">apiVersion: tekton.dev/v1beta1 511 kind: PipelineRun 512 metadata: 513 generateName: run-std-maven- 514 spec: 515 pipelineRef: 516 name: std-maven 517 params: 518 - name: url 519 value: https://github.com/spring-projects/spring-petclinic 520 workspaces: 521 - name: maven-settings 522 configMap: 523 name: custom-maven-settings 524 items: 525 - key: settings.xml 526 path: settings.xml 527 - name: local-maven-repo 528 persistentVolumeClaim: 529 claimName: maven-repo-pvc 530 - name: ws 531 volumeClaimTemplate: 532 spec: 533 accessModes: 534 - ReadWriteMany 535 resources: 536 requests: 537 storage: 1Gi 538 </pre> 539 </div> 540 541 <p> 542 Notes: 543 </p> 544 <ul class="org-ul"> 545 <li>Need <code>affinity-assistant</code> to be disabled (as of today)</li> 546 <li><code>params</code> and <code>workspaces</code> are duplicated in there. 547 <i>Maybe we could be able to specify workspace to be available for all tasks</i></li> 548 </ul> 549 </div> 550 </div> 551 <div id="outline-container-Gradle" class="outline-4"> 552 <h4 id="Gradle"><span class="done DONE">DONE</span> Gradle</h4> 553 <div class="outline-text-4" id="text-Gradle"> 554 <div class='drawer logbook'> 555 <h6>Logbook</h6> 556 <ul class="org-ul"> 557 <li>State “DONE” from “TODO” <span class="timestamp-wrapper"><span class="timestamp">[2020-11-02 Mon 16:56]</span></span></li> 558 </ul> 559 </div> 560 561 <p> 562 A simple <code>gradle</code> project pipeline that build, run test, packages and publish artifacts 563 (jars) to a maven repository. <i>Note: it uses a maven cache (<code>.m2</code>)</i>. This is the same as above 564 but using <code>gradle</code> instead of <code>maven</code>. 565 </p> 566 567 <div class="org-src-container"> 568 <pre class="src src-yaml">apiVersion: tekton.dev/v1beta1 569 kind: Pipeline 570 metadata: 571 name: std-gradle 572 spec: 573 params: 574 - name: url 575 - name: revision 576 default: "" 577 workspaces: 578 - name: ws 579 - name: local-maven-repo 580 - name: maven-settings 581 optional: true 582 tasks: 583 - name: fetch-repository 584 taskRef: 585 name: git-clone 586 bundle: docker.io/vdemeester/tekton-base-git:v0.1 587 workspaces: 588 - name: output 589 workspace: ws 590 params: 591 - name: url 592 value: $(params.url) 593 - name: unit-tests 594 taskRef: 595 bundle: docker.io/vdemeester/tekton-java:v0.1 596 name: gradle 597 runAfter: 598 - fetch-repository 599 workspaces: 600 - name: source 601 workspace: ws 602 - name: maven-repo 603 workspace: local-maven-repo 604 - name: maven-settings 605 workspace: maven-settings 606 params: 607 - name: GOALS 608 value: ["build"] 609 # - name: release-app 610 # taskRef: 611 # bundle: docker.io/vdemeester/tekton-java:v0.1 612 # name: gradle 613 # runAfter: 614 # - unit-tests 615 # workspaces: 616 # - name: source 617 # workspace: ws 618 # - name: maven-repo 619 # workspace: local-maven-repo 620 # - name: maven-settings 621 # workspace: maven-settings 622 # params: 623 # - name: GOALS 624 # value: 625 # - upload 626 # - -DskipTests=true 627 # - -DaltDeploymentRepository=nexus::default::http://nexus:8081/repository/maven-releases/ 628 # - -DaltSnapshotDeploymentRepository=nexus::default::http://nexus:8081/repository/maven-snapshots/q 629 </pre> 630 </div> 631 632 <p> 633 and the run… 634 </p> 635 636 <div class="org-src-container"> 637 <pre class="src src-yaml">apiVersion: tekton.dev/v1beta1 638 kind: PipelineRun 639 metadata: 640 generateName: run-std-gradle- 641 spec: 642 pipelineRef: 643 name: std-gradle 644 params: 645 - name: url 646 value: https://github.com/spring-petclinic/spring-petclinic-kotlin 647 workspaces: 648 - name: maven-settings 649 configMap: 650 name: custom-maven-settings 651 items: 652 - key: settings.xml 653 path: settings.xml 654 - name: local-maven-repo 655 persistentVolumeClaim: 656 claimName: maven-repo-pvc 657 - name: ws 658 volumeClaimTemplate: 659 spec: 660 accessModes: 661 - ReadWriteMany 662 resources: 663 requests: 664 storage: 1Gi 665 </pre> 666 </div> 667 </div> 668 </div> 669 </div> 670 <div id="outline-container-A%20source-to-image%20Pipeline" class="outline-3"> 671 <h3 id="A%20source-to-image%20Pipeline"><span class="done DONE">DONE</span> A source-to-image Pipeline</h3> 672 <div class="outline-text-3" id="text-A%20source-to-image%20Pipeline"> 673 <div class='drawer logbook'> 674 <h6>Logbook</h6> 675 <ul class="org-ul"> 676 <li>State “DONE” from “TODO” <span class="timestamp-wrapper"><span class="timestamp">[2020-11-10 Tue 17:04]</span></span></li> 677 </ul> 678 </div> 679 680 <p> 681 A pipeline that takes a repository with a <code>Dockerfile</code>, builds and pushes an image from it, 682 and deploy it to kubernetes (using deployment/services). 683 </p> 684 685 <p> 686 Let’s first setup a registry 687 </p> 688 689 <div class="org-src-container"> 690 <pre class="src src-shell">TMD=$(mktemp -d) 691 692 # Generate SSL Certificate 693 openssl req -newkey rsa:4096 -nodes -sha256 -keyout "${TMD}"/ca.key -x509 -days 365 \ 694 -out "${TMD}"/ca.crt -subj "/C=FR/ST=IDF/L=Paris/O=Tekton/OU=Catalog/CN=registry" 695 696 # Create a configmap from these certs 697 kubectl create -n "${tns}" configmap sslcert \ 698 --from-file=ca.crt="${TMD}"/ca.crt --from-file=ca.key="${TMD}"/ca.key 699 </pre> 700 </div> 701 702 <div class="org-src-container"> 703 <pre class="src src-yaml">--- 704 apiVersion: apps/v1 705 kind: Deployment 706 metadata: 707 name: registry 708 spec: 709 selector: 710 matchLabels: 711 run: registry 712 replicas: 1 713 template: 714 metadata: 715 labels: 716 run: registry 717 spec: 718 containers: 719 - name: registry 720 image: docker.io/registry:2 721 ports: 722 - containerPort: 5000 723 volumeMounts: 724 - name: sslcert 725 mountPath: /certs 726 env: 727 - name: REGISTRY_HTTP_TLS_CERTIFICATE 728 value: "/certs/ca.crt" 729 - name: REGISTRY_HTTP_TLS_KEY 730 value: "/certs/ca.key" 731 - name: REGISTRY_HTTP_SECRET 732 value: "tekton" 733 volumes: 734 - name: sslcert 735 configMap: 736 defaultMode: 420 737 items: 738 - key: ca.crt 739 path: ca.crt 740 - key: ca.key 741 path: ca.key 742 name: sslcert 743 --- 744 apiVersion: v1 745 kind: Service 746 metadata: 747 name: registry 748 spec: 749 ports: 750 - port: 5000 751 selector: 752 run: registry 753 </pre> 754 </div> 755 </div> 756 <div id="outline-container-buildah" class="outline-4"> 757 <h4 id="buildah"><span class="done DONE">DONE</span> buildah</h4> 758 <div class="outline-text-4" id="text-buildah"> 759 <div class='drawer logbook'> 760 <h6>Logbook</h6> 761 <ul class="org-ul"> 762 <li>State “DONE” from “TODO” <span class="timestamp-wrapper"><span class="timestamp">[2020-11-09 Mon 16:57]</span></span></li> 763 </ul> 764 </div> 765 766 <div class="org-src-container"> 767 <pre class="src src-yaml">--- 768 apiVersion: tekton.dev/v1beta1 769 kind: Pipeline 770 metadata: 771 name: std-source-to-image-buildah 772 spec: 773 params: 774 - name: url 775 - name: revision 776 default: "" 777 - name: image 778 default: "localhost:5000/foo" 779 - name: pushimage 780 default: "localhost:5000/foo" 781 workspaces: 782 - name: ws 783 - name: sslcertdir 784 optional: true 785 tasks: 786 - name: fetch-repository 787 taskRef: 788 name: git-clone 789 #bundle: docker.io/vdemeester/tekton-base-git:v0.1 790 workspaces: 791 - name: output 792 workspace: ws 793 params: 794 - name: url 795 value: $(params.url) 796 - name: build-and-push 797 taskRef: 798 name: buildah 799 #bundle: docker.io/vdemeester/tekton-builders:v0.1 800 runAfter: [ fetch-repository ] 801 params: 802 - name: IMAGE 803 value: $(params.pushimage) 804 - name: TLSVERIFY 805 value: "false" 806 workspaces: 807 - name: source 808 workspace: ws 809 # - name: sslcertdir 810 # workspace: sslcertdir 811 - name: deploy 812 runAfter: [ build-and-push ] 813 params: 814 - name: reference 815 value: $(params.image)@$(tasks.build-and-push.results.IMAGE_DIGEST) 816 taskSpec: 817 params: 818 - name: reference 819 steps: 820 - image: gcr.io/cloud-builders/kubectl@sha256:8ab94be8b2b4f3d117f02d868b39540fddd225447abf4014f7ba4765cb39f753 821 script: | 822 cat <<EOF | kubectl apply -f - 823 apiVersion: apps/v1 824 kind: Deployment 825 metadata: 826 name: foo-app 827 spec: 828 selector: 829 matchLabels: 830 run: foo-app 831 replicas: 1 832 template: 833 metadata: 834 labels: 835 run: foo-app 836 spec: 837 containers: 838 - name: foo 839 image: $(params.reference) 840 </pre> 841 </div> 842 843 <div class="org-src-container"> 844 <pre class="src src-yaml">apiVersion: tekton.dev/v1beta1 845 kind: PipelineRun 846 metadata: 847 generateName: run-std-source-to-image-buildah- 848 spec: 849 pipelineRef: 850 name: std-source-to-image-buildah 851 params: 852 - name: url 853 value: https://github.com/lvthillo/python-flask-docker 854 - name: pushimage 855 value: sakhalin.home:5000/foo 856 workspaces: 857 - name: ws 858 volumeClaimTemplate: 859 spec: 860 accessModes: 861 - ReadWriteOnce 862 resources: 863 requests: 864 storage: 1Gi 865 </pre> 866 </div> 867 868 <p> 869 Notes: 870 </p> 871 <ul class="org-ul"> 872 <li><code>deploy</code> may need it’s own task definition in the catalog. <code>kubectl-deploy-pod</code> is one but 873 didn’t work properly</li> 874 <li>rest is smooth</li> 875 </ul> 876 </div> 877 </div> 878 <div id="outline-container-s2i%20%28no%20%3DDockerfile%3D%29" class="outline-4"> 879 <h4 id="s2i%20%28no%20%3DDockerfile%3D%29"><span class="done DONE">DONE</span> s2i (no <code>Dockerfile</code>)</h4> 880 <div class="outline-text-4" id="text-s2i%20%28no%20%3DDockerfile%3D%29"> 881 <div class='drawer logbook'> 882 <h6>Logbook</h6> 883 <ul class="org-ul"> 884 <li>State “DONE” from “TODO” <span class="timestamp-wrapper"><span class="timestamp">[2020-11-10 Tue 16:59]</span></span></li> 885 </ul> 886 </div> 887 888 <div class="org-src-container"> 889 <pre class="src src-yaml">--- 890 apiVersion: tekton.dev/v1beta1 891 kind: Pipeline 892 metadata: 893 name: std-source-to-image-s2i 894 spec: 895 params: 896 - name: url 897 - name: revision 898 default: "" 899 - name: image 900 default: "localhost:5000/foo" 901 - name: pushimage 902 default: "localhost:5000/foo" 903 workspaces: 904 - name: ws 905 - name: sslcertdir 906 optional: true 907 tasks: 908 - name: fetch-repository 909 taskRef: 910 name: git-clone 911 #bundle: docker.io/vdemeester/tekton-base-git:v0.1 912 workspaces: 913 - name: output 914 workspace: ws 915 params: 916 - name: url 917 value: $(params.url) 918 - name: build-and-push 919 taskRef: 920 name: s2i 921 #bundle: docker.io/vdemeester/tekton-builders:v0.1 922 runAfter: [ fetch-repository ] 923 params: 924 - name: BUILDER_IMAGE 925 value: docker.io/fabric8/s2i-java:latest-java11 926 - name: S2I_EXTRA_ARGS 927 value: "--image-scripts-url=image:///usr/local/s2i" 928 - name: IMAGE 929 value: $(params.pushimage) 930 - name: TLSVERIFY 931 value: "false" 932 workspaces: 933 - name: source 934 workspace: ws 935 # - name: sslcertdir 936 # workspace: sslcertdir 937 - name: deploy 938 runAfter: [ build-and-push ] 939 params: 940 - name: reference 941 value: $(params.image)@$(tasks.build-and-push.results.IMAGE_DIGEST) 942 taskSpec: 943 params: 944 - name: reference 945 steps: 946 - image: gcr.io/cloud-builders/kubectl@sha256:8ab94be8b2b4f3d117f02d868b39540fddd225447abf4014f7ba4765cb39f753 947 script: | 948 cat <<EOF | kubectl apply -f - 949 apiVersion: apps/v1 950 kind: Deployment 951 metadata: 952 name: foo-app 953 spec: 954 selector: 955 matchLabels: 956 run: foo-app 957 replicas: 1 958 template: 959 metadata: 960 labels: 961 run: foo-app 962 spec: 963 containers: 964 - name: foo 965 image: $(params.reference) 966 </pre> 967 </div> 968 969 <div class="org-src-container"> 970 <pre class="src src-yaml">apiVersion: tekton.dev/v1beta1 971 kind: PipelineRun 972 metadata: 973 generateName: run-std-source-to-image-s2i- 974 spec: 975 pipelineRef: 976 name: std-source-to-image-s2i 977 params: 978 - name: url 979 value: https://github.com/siamaksade/spring-petclinic 980 - name: pushimage 981 value: sakhalin.home:5000/foo 982 workspaces: 983 - name: ws 984 volumeClaimTemplate: 985 spec: 986 accessModes: 987 - ReadWriteOnce 988 resources: 989 requests: 990 storage: 1Gi 991 </pre> 992 </div> 993 994 <p> 995 Notes: 996 </p> 997 <ul class="org-ul"> 998 <li><code>s2i</code> shares a lot with <code>buildah</code> or any <code>Dockerfile</code> build tool. 999 This may <b>show</b> the need to compose tasks from other tasks. Here we do <code>s2i … 1000 --as-dockerfile</code> and then we just need to build the <code>Dockerfile</code>. This could be 2 separate 1001 tasks but it would make the pipeline less efficient.</li> 1002 </ul> 1003 </div> 1004 </div> 1005 </div> 1006 <div id="outline-container-A%20source-to-image%20%22knative%22%20Pipeline" class="outline-3"> 1007 <h3 id="A%20source-to-image%20%22knative%22%20Pipeline"><span class="done DONE">DONE</span> A source-to-image “knative” Pipeline</h3> 1008 <div class="outline-text-3" id="text-A%20source-to-image%20%22knative%22%20Pipeline"> 1009 <div class='drawer logbook'> 1010 <h6>Logbook</h6> 1011 <ul class="org-ul"> 1012 <li>State “DONE” from “TODO” <span class="timestamp-wrapper"><span class="timestamp">[2020-11-10 Tue 17:02]</span></span></li> 1013 </ul> 1014 </div> 1015 1016 <p> 1017 A pipeline that takes a repository with a <code>Dockerfile</code>, builds and pushes an image from it, 1018 and deploy it to kubernetes using knative services. 1019 </p> 1020 1021 <div class="org-src-container"> 1022 <pre class="src src-yaml">--- 1023 apiVersion: tekton.dev/v1beta1 1024 kind: Pipeline 1025 metadata: 1026 name: std-source-to-image-buildah-kn 1027 spec: 1028 params: 1029 - name: url 1030 - name: revision 1031 default: "" 1032 - name: image 1033 default: "localhost:5000/foo" 1034 - name: pushimage 1035 default: "localhost:5000/foo" 1036 workspaces: 1037 - name: ws 1038 - name: sslcertdir 1039 optional: true 1040 tasks: 1041 - name: fetch-repository 1042 taskRef: 1043 name: git-clone 1044 #bundle: docker.io/vdemeester/tekton-base-git:v0.1 1045 workspaces: 1046 - name: output 1047 workspace: ws 1048 params: 1049 - name: url 1050 value: $(params.url) 1051 - name: build-and-push 1052 taskRef: 1053 name: buildah 1054 #bundle: docker.io/vdemeester/tekton-builders:v0.1 1055 runAfter: [ fetch-repository ] 1056 params: 1057 - name: IMAGE 1058 value: $(params.pushimage) 1059 - name: TLSVERIFY 1060 value: "false" 1061 workspaces: 1062 - name: source 1063 workspace: ws 1064 # - name: sslcertdir 1065 # workspace: sslcertdir 1066 - name: kn-deploy 1067 runAfter: [ build-and-push ] 1068 taskref: 1069 name: kn 1070 params: 1071 - name: ARGS 1072 value: 1073 - "service" 1074 - "create" 1075 - "hello" 1076 - "--force" 1077 - "--image=$(params.image)@$(tasks.build-and-push.results.IMAGE_DIGEST)" 1078 </pre> 1079 </div> 1080 1081 <div class="org-src-container"> 1082 <pre class="src src-yaml">apiVersion: tekton.dev/v1beta1 1083 kind: PipelineRun 1084 metadata: 1085 generateName: run-std-source-to-image-buildah-kn- 1086 spec: 1087 pipelineRef: 1088 name: std-source-to-image-buildah-kn 1089 params: 1090 - name: url 1091 value: https://github.com/lvthillo/python-flask-docker 1092 - name: pushimage 1093 value: sakhalin.home:5000/foo 1094 serviceAccountName: kn-deployer-account 1095 workspaces: 1096 - name: ws 1097 volumeClaimTemplate: 1098 spec: 1099 accessModes: 1100 - ReadWriteOnce 1101 resources: 1102 requests: 1103 storage: 1Gi 1104 </pre> 1105 </div> 1106 </div> 1107 </div> 1108 <div id="outline-container-A%20canary%20deployment%20pipeline%20%28not%20from%20sources%29" class="outline-3"> 1109 <h3 id="A%20canary%20deployment%20pipeline%20%28not%20from%20sources%29"><span class="todo TODO">TODO</span> A canary deployment pipeline (not from sources)</h3> 1110 </div> 1111 1112 1113 1114 <div id="outline-container-A%20canary%20deployment%20pipeline%20%28iter8%29" class="outline-3"> 1115 <h3 id="A%20canary%20deployment%20pipeline%20%28iter8%29"><span class="todo TODO">TODO</span> A canary deployment pipeline (iter8)</h3> 1116 <div class="outline-text-3" id="text-A%20canary%20deployment%20pipeline%20%28iter8%29"> 1117 <p> 1118 This is taken from <a href="https://github.com/iter8-tools/canary-tekton-example">iter8 canary tekton example</a>. 1119 </p> 1120 1121 1122 <figure id="org2a3fc60"> 1123 <img src="./images/tekton/canary-pipeline.png" alt="canary-pipeline.png"> 1124 1125 </figure> 1126 1127 <div class="org-src-container"> 1128 <pre class="src src-yaml">apiVersion: tekton.dev/v1beta1 1129 kind: Task 1130 metadata: 1131 name: identify-baseline-task 1132 spec: 1133 description: | 1134 Identify the baseline deployment in a cluster namespace. 1135 params: 1136 - name: UID 1137 type: string 1138 default: "uid" 1139 description: | 1140 Unique identifier used to assocaite load with an experiment. 1141 Suitable values might be the experiment name of the task/pipeline run name/uid. 1142 - name: NAMESPACE 1143 type: string 1144 default: default 1145 description: The cluster namespace in which to search for the baseline. 1146 - name: EXPERIMENT_TEMPLATE 1147 type: string 1148 default: "experiment" 1149 description: Name of template that should be used for the experiment. 1150 workspaces: 1151 - name: source 1152 results: 1153 - name: baseline 1154 description: Name of the baseline deployment. 1155 steps: 1156 - name: update-experiment 1157 workingDir: $(workspaces.source.path)/$(params.UID) 1158 image: kalantar/yq-kubernetes 1159 script: | 1160 #!/usr/bin/env bash 1161 # Uncomment to debug 1162 set -x 1163 1164 # Identify baseline deployment for an experiment 1165 # This is heuristic; prefers to look at stable DestinationRule 1166 # But if this isn't defined will select first deployment that satisfies 1167 # the service selector (service from Experiment) 1168 1169 NAMESPACE=$(params.NAMESPACE) 1170 SERVICE=$(yq read $(params.EXPERIMENT_TEMPLATE) spec.service.name) 1171 ROUTER=$(yq read $(params.EXPERIMENT_TEMPLATE) spec.networking.id) 1172 1173 if [[ -z ${ROUTER} ]] || [[ "${ROUTER}" == "null" ]]; then 1174 ROUTER="${SERVICE}.${NAMESPACE}.svc.cluster.local" 1175 fi 1176 1177 echo "SERVICE=${SERVICE}" 1178 echo " ROUTER=${ROUTER}" 1179 1180 SUBSET= 1181 NUM_VS=$(kubectl --namespace ${NAMESPACE} get vs --selector=iter8-tools/router=${ROUTER} --output json | jq '.items | length') 1182 echo "NUM_VS=${NUM_VS}" 1183 if (( ${NUM_VS} > 0 )); then 1184 SUBSET=$(kubectl --namespace ${NAMESPACE} get vs --selector=iter8-tools/router=${ROUTER} --output json | jq -r '.items[0].spec.http[0].route[] | select(has("weight")) | select(.weight == 100) | .destination.subset') 1185 echo "SUBSET=$SUBSET" 1186 fi 1187 1188 DEPLOY_SELECTOR="" 1189 if [[ -n ${SUBSET} ]]; then 1190 NUM_DR=$(kubectl --namespace ${NAMESPACE} get dr --selector=iter8-tools/router=${ROUTER} --output json | jq '.items | length') 1191 echo "NUM_DR=${NUM_DR}" 1192 if (( ${NUM_DR} > 0 )); then 1193 DEPLOY_SELECTOR=$(kubectl --namespace ${NAMESPACE} get dr --selector=iter8-tools/router=${ROUTER} --output json | jq -r --arg SUBSET "$SUBSET" '.items[0].spec.subsets[] | select(.name == $SUBSET) | .labels | to_entries[] | "\(.key)=\(.value)"' | paste -sd',' -) 1194 fi 1195 fi 1196 echo "DEPLOY_SELECTOR=${DEPLOY_SELECTOR}" 1197 1198 if [ -z "${DEPLOY_SELECTOR}" ]; then 1199 # No stable DestinationRule found so find the deployment(s) implementing $SERVICE 1200 DEPLOY_SELECTOR=$(kubectl --namespace ${NAMESPACE} get service ${SERVICE} --output json | jq -r '.spec.selector | to_entries[] | "\(.key)=\(.value)"' | paste -sd',' -) 1201 fi 1202 echo "DEPLOY_SELECTOR=$DEPLOY_SELECTOR" 1203 1204 NUM_DEPLOY=$(kubectl --namespace ${NAMESPACE} get deployment --selector=${DEPLOY_SELECTOR} --output json | jq '.items | length') 1205 echo " NUM_DEPLOY=${NUM_DEPLOY}" 1206 BASELINE_DEPLOYMENT_NAME= 1207 if (( ${NUM_DEPLOY} > 0 )); then 1208 BASELINE_DEPLOYMENT_NAME=$(kubectl --namespace ${NAMESPACE} get deployment --selector=${DEPLOY_SELECTOR} --output jsonpath='{.items[0].metadata.name}') 1209 fi 1210 echo -n "${BASELINE_DEPLOYMENT_NAME}" | tee $(results.baseline.path) 1211 --- 1212 apiVersion: tekton.dev/v1beta1 1213 kind: Task 1214 metadata: 1215 name: define-experiment-task 1216 spec: 1217 description: | 1218 Define an iter8 canary Experiment from a template. 1219 workspaces: 1220 - name: source 1221 description: Consisting of kubernetes manifest templates (ie, the Experiment) 1222 params: 1223 - name: UID 1224 default: "uid" 1225 description: | 1226 Unique identifier used to assocaite load with an experiment. 1227 Suitable values might be the experiment name of the task/pipeline run name/uid. 1228 - name: EXPERIMENT_TEMPLATE 1229 type: string 1230 default: "experiment.yaml" 1231 description: An experiment resource that can be modified. 1232 - name: NAME 1233 type: string 1234 default: "" 1235 description: The name of the experiment resource to create 1236 - name: BASELINE 1237 type: string 1238 default: "" 1239 description: The name of the baseline resource 1240 - name: CANDIDATE 1241 type: string 1242 default: "" 1243 description: The name of the candidate (canary) resource 1244 results: 1245 - name: experiment 1246 description: Path to experiment (in workspace ) 1247 steps: 1248 - name: update-experiment 1249 image: kalantar/yq-kubernetes 1250 workingDir: $(workspaces.source.path)/$(params.UID) 1251 script: | 1252 #!/usr/bin/env bash 1253 1254 OUTPUT="experiment-$(params.UID).yaml" 1255 1256 if [ -f "$(params.EXPERIMENT_TEMPLATE)" ]; then 1257 cp "$(params.EXPERIMENT_TEMPLATE)" "${OUTPUT}" 1258 else 1259 curl -s -o "${OUTPUT}" "$(params.EXPERIMENT_TEMPLATE)" 1260 fi 1261 1262 if [ ! -f "${OUTPUT}" ]; then 1263 echo "Can not read template: $(params.EXPERIMENT_TEMPLATE)" 1264 exit 1 1265 fi 1266 1267 # Update experiment template 1268 if [ "" != "$(params.NAME)" ]; then 1269 yq write --inplace "${OUTPUT}" metadata.name "$(params.NAME)" 1270 fi 1271 if [ "" != "$(params.BASELINE)" ]; then 1272 yq write --inplace "${OUTPUT}" spec.service.baseline "$(params.BASELINE)" 1273 fi 1274 if [ "" != "$(params.CANDIDATE)" ]; then 1275 yq write --inplace "${OUTPUT}" spec.service.candidates[0] "$(params.CANDIDATE)" 1276 fi 1277 1278 cat "${OUTPUT}" 1279 echo -n $(params.UID)/${OUTPUT} | tee $(results.experiment.path) 1280 --- 1281 apiVersion: tekton.dev/v1beta1 1282 kind: Task 1283 metadata: 1284 name: apply-manifest-task 1285 spec: 1286 description: | 1287 Create an iter8 canary Experiment from a template. 1288 workspaces: 1289 - name: manifest-dir 1290 description: Consisting of kubernetes manifests (ie, the Experiment) 1291 params: 1292 - name: MANIFEST 1293 type: string 1294 default: "manifest.yaml" 1295 description: The name of the file containing the kubernetes manifest to apply 1296 - name: TARGET_NAMESPACE 1297 type: string 1298 default: "default" 1299 description: The namespace in which the manifest should be applied 1300 steps: 1301 - name: apply-manifest 1302 image: kalantar/yq-kubernetes 1303 workingDir: $(workspaces.manifest-dir.path) 1304 script: | 1305 #!/usr/bin/env bash 1306 1307 # Create experiment in cluster 1308 kubectl --namespace $(params.TARGET_NAMESPACE) apply --filename "$(params.MANIFEST)" 1309 --- 1310 apiVersion: tekton.dev/v1beta1 1311 kind: Task 1312 metadata: 1313 name: define-canary-task 1314 spec: 1315 description: | 1316 Create YAML file needed to deploy the canary version of the application. 1317 Relies on kustomize and assumes a patch file template (PATCH_FILE) containing the keyword 1318 "VERSION" that can be replaced with the canary verion. 1319 params: 1320 - name: UID 1321 default: "uid" 1322 description: | 1323 Unique identifier used to assocaite load with an experiment. 1324 Suitable values might be the experiment name of the task/pipeline run name/uid. 1325 - name: image-repository 1326 description: Docker image repository 1327 default: "" 1328 - name: image-tag 1329 description: tag of image to deploy 1330 default: latest 1331 - name: PATCH_FILE 1332 default: kustomize/patch.yaml 1333 workspaces: 1334 - name: source 1335 results: 1336 - name: deployment-file 1337 description: Path to file (in workspace ) 1338 1339 steps: 1340 - name: modify-patch 1341 image: alpine 1342 workingDir: $(workspaces.source.path)/$(params.UID) 1343 script: | 1344 #!/usr/bin/env sh 1345 1346 IMAGE_TAG=$(params.image-tag) 1347 PATCH_FILE=$(params.PATCH_FILE) 1348 IMAGE=$(params.image-repository):$(params.image-tag) 1349 1350 sed -i -e "s#iter8/reviews:istio-VERSION#${IMAGE}#" ${PATCH_FILE} 1351 sed -i -e "s#VERSION#${IMAGE_TAG}#g" ${PATCH_FILE} 1352 cat ${PATCH_FILE} 1353 1354 echo -n "deploy-$(params.UID).yaml" | tee $(results.deployment-file.path) 1355 1356 - name: generate-deployment 1357 image: smartive/kustomize 1358 workingDir: $(workspaces.source.path)/$(params.UID) 1359 command: [ "kustomize" ] 1360 args: [ "build", "kustomize", "-o", "deploy-$(params.UID).yaml" ] 1361 1362 - name: log-deployment 1363 image: alpine 1364 workingDir: $(workspaces.source.path)/$(params.UID) 1365 command: [ "cat" ] 1366 args: [ "deploy-$(params.UID).yaml" ] 1367 --- 1368 apiVersion: tekton.dev/v1beta1 1369 kind: Task 1370 metadata: 1371 name: wait-completion-task 1372 spec: 1373 description: | 1374 Wait until EXPERIMENT is completed; 1375 that is, condition ExperimentCompleted is true. 1376 params: 1377 - name: EXPERIMENT 1378 default: "experiment" 1379 description: Name of iter8 experiment 1380 - name: NAMESPACE 1381 default: default 1382 description: Namespace in which the iter8 experiment is defined. 1383 - name: TIMEOUT 1384 default: "1h" 1385 description: Amount of time to wait for experiment to complete. 1386 steps: 1387 - name: wait 1388 image: kalantar/yq-kubectl 1389 script: | 1390 #!/usr/bin/env sh 1391 set -x 1392 1393 kubectl --namespace $(params.NAMESPACE) wait \ 1394 --for=condition=ExperimentCompleted \ 1395 experiments.iter8.tools $(params.EXPERIMENT) \ 1396 --timeout=$(params.TIMEOUT) 1397 --- 1398 apiVersion: tekton.dev/v1beta1 1399 kind: Task 1400 metadata: 1401 name: cleanup-task 1402 spec: 1403 workspaces: 1404 - name: workspace 1405 params: 1406 - name: UID 1407 default: "uid" 1408 description: | 1409 Unique identifier used to assocaite load with an experiment. 1410 Suitable values might be the experiment name of the task/pipeline run name/uid. 1411 steps: 1412 - name: clean-workspace 1413 image: alpine 1414 script: | 1415 #!/usr/bin/env sh 1416 set -x 1417 1418 rm -rf $(workspaces.workspace.path)/$(params.UID) 1419 --- 1420 apiVersion: tekton.dev/v1beta1 1421 kind: Task 1422 metadata: 1423 name: identify-endpoint-task 1424 spec: 1425 description: | 1426 Identify URL of application to be used buy load generator. 1427 params: 1428 - name: istio-namespace 1429 default: istio-system 1430 description: Namespace where Istio is installed. 1431 - name: application-query 1432 default: "" 1433 description: Application endpoint. 1434 results: 1435 - name: application-url 1436 description: The URL that can be used to apply load to the application. 1437 steps: 1438 - name: determine-server 1439 image: kalantar/yq-kubernetes 1440 script: | 1441 #!/usr/bin/env sh 1442 1443 # Determine the IP 1444 # Try loadbalancer on istio-ingressgateway 1445 IP=$(kubectl --namespace $(params.istio-namespace) get service istio-ingressgateway --output jsonpath='{.status.loadBalancer.ingress[0].ip}') 1446 # If not, try an external IP for a node 1447 echo "IP=${IP}" 1448 if [ -z "${IP}" ]; then 1449 IP=$(kubectl get nodes -o jsonpath='{.items[0].status.addresses[?(@.type == "ExternalIP")].address}') 1450 fi 1451 echo "IP=${IP}" 1452 # If not, try an internal IP for a node (minikube) 1453 if [ -z "${IP}" ]; then 1454 IP=$(kubectl get nodes -o jsonpath='{.items[0].status.addresses[?(@.type == "InternalIP")].address}') 1455 fi 1456 echo "IP=${IP}" 1457 1458 # Determine the port 1459 PORT=$(kubectl --namespace $(params.istio-namespace) get service istio-ingressgateway --output jsonpath="{.spec.ports[?(@.port==80)].nodePort}") 1460 echo "PORT=${PORT}" 1461 1462 HOST="${IP}:${PORT}" 1463 echo "HOST=$HOST" 1464 1465 echo -n "http://${HOST}/$(params.application-query)" | tee $(results.application-url.path) 1466 --- 1467 apiVersion: tekton.dev/v1beta1 1468 kind: Task 1469 metadata: 1470 name: generate-load-task 1471 spec: 1472 description: | 1473 Generate load by sending queries to URL every INTERVAL seconds. 1474 Load generation continues as long as the file terminate is not present. 1475 params: 1476 - name: UID 1477 default: "uid" 1478 description: | 1479 Unique identifier used to assocaite load with an experiment. 1480 Suitable values might be the experiment name of the task/pipeline run name/uid. 1481 - name: URL 1482 default: "http://localhost:8080" 1483 description: URL that should be used to generate load. 1484 - name: HOST 1485 default: "" 1486 description: Value to be added in Host header. 1487 - name: terminate 1488 default: ".terminate" 1489 description: Name of file that, if present, triggers termination of load generation. 1490 - name: INTERVAL 1491 default: "0.1" 1492 description: Interval (s) between generated requests. 1493 workspaces: 1494 - name: scratch 1495 steps: 1496 - name: generate-load 1497 image: kalantar/yq-kubernetes 1498 workingDir: $(workspaces.scratch.path) 1499 script: | 1500 #!/usr/bin/env bash 1501 1502 # Remove terminatation file if it exists (it should not) 1503 rm -f $(params.UID)/$(params.terminate) || true 1504 1505 echo "param HOST=$(params.HOST)" 1506 echo "param URL=$(params.URL)" 1507 1508 if [ "$(params.HOST)" == "" ]; then 1509 HOST= 1510 elif [ "$(params.HOST)" == "\*" ]; then 1511 HOST= 1512 else 1513 HOST=$(params.HOST) 1514 fi 1515 echo "computed HOST=$HOST" 1516 1517 # Optionally use a Host header in requests 1518 if [ -z ${HOST} ]; then 1519 echo "curl -o /dev/null -s -w \"%{http_code}\\n\" $(params.URL)" 1520 else 1521 echo "curl -H \"Host: ${HOST}\" -o /dev/null -s -w \"%{http_code}\\n\" $(params.URL)" 1522 fi 1523 1524 # Generate load until the file terminate is created. 1525 REQUESTS=0 1526 ERRORS=0 1527 while [ 1 ]; do 1528 if [ -f $(params.UID)/$(params.terminate) ]; then 1529 echo "Terminating load; ${REQUESTS} requests sent; ${ERRORS} had errors." 1530 break 1531 fi 1532 sleep $(params.INTERVAL) 1533 OUT= 1534 if [ -z ${HOST} ]; then 1535 OUT=$(curl -o /dev/null -s -w "%{http_code}\n" $(params.URL)) 1536 else 1537 OUT=$(curl -H "Host: ${HOST}" -o /dev/null -s -w "%{http_code}\n" $(params.URL)) 1538 fi 1539 if [ "${OUT}" != "200" ]; then ((ERRORS++)); echo "Not OK: ${OUT}"; fi 1540 ((REQUESTS++)) 1541 done 1542 --- 1543 apiVersion: tekton.dev/v1beta1 1544 kind: Task 1545 metadata: 1546 name: stop-load-task 1547 spec: 1548 description: | 1549 Trigger the termination of experiment load. 1550 params: 1551 - name: UID 1552 default: "uid" 1553 description: | 1554 Unique identifier used to assocaite load with an experiment. 1555 Suitable values might be the experiment name of the task/pipeline run name/uid. 1556 - name: terminate 1557 default: ".terminate" 1558 description: Name of file that, if present, triggers termination of load generation. 1559 workspaces: 1560 - name: scratch 1561 steps: 1562 - name: wait 1563 image: alpine 1564 workingDir: $(workspaces.scratch.path) 1565 script: | 1566 #!/usr/bin/env sh 1567 1568 # To avoid conflicts, use a run specific subdirectory 1569 mkdir -p $(params.UID) 1570 touch $(params.UID)/$(params.terminate) 1571 --- 1572 apiVersion: tekton.dev/v1beta1 1573 kind: Task 1574 metadata: 1575 name: queue-request-task 1576 spec: 1577 description: | 1578 Place self at the end of a queue and wait until we are at the top. 1579 params: 1580 - name: UID 1581 default: "uid" 1582 description: | 1583 Unique identifier used to assocaite load with an experiment. 1584 Suitable values might be the experiment name of the task/pipeline run name/uid. 1585 - name: lock-dir 1586 default: ".lock" 1587 description: Name of directory to use to acquire mutex. 1588 - name: queue 1589 default: ".queue" 1590 description: Name of the file containing execution queue. 1591 - name: wait-time 1592 default: "20" 1593 description: Sleep time between attempts to aquire the lock. 1594 workspaces: 1595 - name: scratch 1596 steps: 1597 - name: queue 1598 image: alpine 1599 workingDir: $(workspaces.scratch.path) 1600 script: | 1601 #!/usr/bin/env sh 1602 1603 while [ "$(params.UID)" != "$(tail -n 1 $(params.queue))" ]; do 1604 if mkdir "$(params.lock-dir)"; then 1605 echo "queuing $(params.UID)" 1606 echo $(params.UID) >> $(params.queue) 1607 rm -rf "$(params.lock-dir)" 1608 else 1609 sleep $(params.wait-time) 1610 fi 1611 done 1612 - name: wait-head 1613 image: alpine 1614 workingDir: $(workspaces.scratch.path) 1615 script: | 1616 #!/usr/bin/env sh 1617 1618 while [ "$(params.UID)" != "$(head -n 1 $(params.queue))" ]; do 1619 sleep $(params.wait-time) 1620 done 1621 echo "$(params.UID) proceeding" 1622 --- 1623 apiVersion: tekton.dev/v1beta1 1624 kind: Task 1625 metadata: 1626 name: dequeue-request-task 1627 spec: 1628 description: | 1629 Remove entry from top of queue. 1630 params: 1631 - name: queue 1632 default: ".queue" 1633 description: Name of the file containing execution queue. 1634 workspaces: 1635 - name: scratch 1636 steps: 1637 - name: dequeue 1638 image: alpine 1639 workingDir: $(workspaces.scratch.path) 1640 script: | 1641 #!/usr/bin/env sh 1642 1643 tail -n +2 $(params.queue) > /tmp/$$; mv /tmp/$$ $(params.queue) 1644 </pre> 1645 </div> 1646 1647 <div class="org-src-container"> 1648 <pre class="src src-yaml">apiVersion: tekton.dev/v1beta1 1649 kind: Pipeline 1650 metadata: 1651 name: canary-rollout-iter8 1652 spec: 1653 workspaces: 1654 - name: source 1655 - name: experiment-dir 1656 params: 1657 - name: application-source 1658 type: string 1659 description: URL of source git repository. 1660 default: "" 1661 - name: application-namespace 1662 type: string 1663 description: Target namespace for application. 1664 - name: application-query 1665 type: string 1666 description: Service query for load generation. 1667 default: "" 1668 - name: application-image 1669 type: string 1670 description: Docker image repository for image to deploy. 1671 - name: HOST 1672 type: string 1673 description: Value that should be sent in Host header in test queries 1674 default: "" 1675 1676 - name: experiment 1677 type: string 1678 description: Name of experiment to create. 1679 default: "experiment" 1680 - name: experiment-template 1681 type: string 1682 description: Template for experiment to create. 1683 1684 - name: terminate 1685 type: string 1686 default: ".terminate" 1687 description: Name of file that, if present, triggers termination of load generation. 1688 1689 tasks: 1690 - name: initialize-request 1691 taskRef: 1692 name: queue-request-task 1693 workspaces: 1694 - name: scratch 1695 workspace: experiment-dir 1696 params: 1697 - name: UID 1698 value: $(context.pipelineRun.uid) 1699 1700 - name: clone-source 1701 taskRef: 1702 name: git-clone 1703 runAfter: 1704 - initialize-request 1705 workspaces: 1706 - name: output 1707 workspace: source 1708 params: 1709 - name: url 1710 value: $(params.application-source) 1711 - name: revision 1712 value: master 1713 - name: deleteExisting 1714 value: "true" 1715 - name: subdirectory 1716 value: $(context.pipelineRun.uid) 1717 1718 - name: build-and-push-image 1719 taskRef: 1720 name: kaniko 1721 runAfter: 1722 - clone-source 1723 timeout: "15m" 1724 workspaces: 1725 - name: source 1726 workspace: source 1727 params: 1728 - name: DOCKERFILE 1729 value: ./$(context.pipelineRun.uid)/Dockerfile 1730 - name: CONTEXT 1731 value: ./$(context.pipelineRun.uid) 1732 - name: IMAGE 1733 value: $(params.application-image):$(tasks.clone-source.results.commit) 1734 - name: EXTRA_ARGS 1735 value: "--skip-tls-verify" 1736 1737 - name: identify-baseline 1738 taskRef: 1739 name: identify-baseline-task 1740 runAfter: 1741 - clone-source 1742 workspaces: 1743 - name: source 1744 workspace: source 1745 params: 1746 - name: UID 1747 value: $(context.pipelineRun.uid) 1748 - name: NAMESPACE 1749 value: $(params.application-namespace) 1750 - name: EXPERIMENT_TEMPLATE 1751 value: $(params.experiment-template) 1752 1753 - name: define-experiment 1754 taskRef: 1755 name: define-experiment-task 1756 runAfter: 1757 - clone-source 1758 - identify-baseline 1759 workspaces: 1760 - name: source 1761 workspace: source 1762 params: 1763 - name: UID 1764 value: $(context.pipelineRun.uid) 1765 - name: EXPERIMENT_TEMPLATE 1766 value: $(params.experiment-template) 1767 - name: NAME 1768 value: $(context.pipelineRun.uid) 1769 - name: BASELINE 1770 value: $(tasks.identify-baseline.results.baseline) 1771 - name: CANDIDATE 1772 value: reviews-$(tasks.clone-source.results.commit) 1773 1774 - name: create-experiment 1775 taskRef: 1776 name: apply-manifest-task 1777 runAfter: 1778 - define-experiment 1779 workspaces: 1780 - name: manifest-dir 1781 workspace: source 1782 params: 1783 - name: TARGET_NAMESPACE 1784 value: $(params.application-namespace) 1785 - name: MANIFEST 1786 value: $(tasks.define-experiment.results.experiment) 1787 1788 - name: define-canary 1789 taskRef: 1790 name: define-canary-task 1791 runAfter: 1792 - clone-source 1793 workspaces: 1794 - name: source 1795 workspace: source 1796 params: 1797 - name: UID 1798 value: $(context.pipelineRun.uid) 1799 - name: image-repository 1800 value: $(params.application-image) 1801 - name: image-tag 1802 value: $(tasks.clone-source.results.commit) 1803 1804 - name: deploy-canary 1805 taskRef: 1806 name: apply-manifest-task 1807 runAfter: 1808 - create-experiment 1809 - build-and-push-image 1810 - define-canary 1811 workspaces: 1812 - name: manifest-dir 1813 workspace: source 1814 params: 1815 - name: TARGET_NAMESPACE 1816 value: $(params.application-namespace) 1817 - name: MANIFEST 1818 value: $(context.pipelineRun.uid)/$(tasks.define-canary.results.deployment-file) 1819 1820 - name: identify-endpoint 1821 taskRef: 1822 name: identify-endpoint-task 1823 runAfter: 1824 - initialize-request 1825 params: 1826 - name: application-query 1827 value: $(params.application-query) 1828 1829 - name: generate-load 1830 taskRef: 1831 name: generate-load-task 1832 runAfter: 1833 - create-experiment 1834 - identify-endpoint 1835 workspaces: 1836 - name: scratch 1837 workspace: experiment-dir 1838 params: 1839 - name: UID 1840 value: $(context.pipelineRun.uid) 1841 - name: URL 1842 value: $(tasks.identify-endpoint.results.application-url) 1843 - name: HOST 1844 value: $(params.HOST) 1845 - name: terminate 1846 value: $(params.terminate) 1847 1848 - name: wait-completion 1849 taskRef: 1850 name: wait-completion-task 1851 runAfter: 1852 - deploy-canary 1853 params: 1854 - name: EXPERIMENT 1855 value: $(context.pipelineRun.uid) 1856 - name: NAMESPACE 1857 value: $(params.application-namespace) 1858 1859 - name: stop-load-generation 1860 runAfter: 1861 - wait-completion 1862 taskRef: 1863 name: stop-load-task 1864 workspaces: 1865 - name: scratch 1866 workspace: experiment-dir 1867 params: 1868 - name: UID 1869 value: $(context.pipelineRun.uid) 1870 - name: terminate 1871 value: $(params.terminate) 1872 1873 finally: 1874 - name: cleanup-scratch-workspace 1875 taskRef: 1876 name: cleanup-task 1877 workspaces: 1878 - name: workspace 1879 workspace: experiment-dir 1880 params: 1881 - name: UID 1882 value: $(context.pipelineRun.uid) 1883 - name: cleanup-source-workspace 1884 taskRef: 1885 name: cleanup-task 1886 workspaces: 1887 - name: workspace 1888 workspace: source 1889 params: 1890 - name: UID 1891 value: $(context.pipelineRun.uid) 1892 - name: complete-request 1893 taskRef: 1894 name: dequeue-request-task 1895 workspaces: 1896 - name: scratch 1897 workspace: experiment-dir 1898 </pre> 1899 </div> 1900 1901 <div class="org-src-container"> 1902 <pre class="src src-yaml">apiVersion: tekton.dev/v1beta1 1903 kind: PipelineRun 1904 metadata: 1905 name: canary-rollout 1906 spec: 1907 pipelineRef: 1908 name: canary-rollout-iter8 1909 serviceAccountName: default 1910 workspaces: 1911 - name: source 1912 persistentVolumeClaim: 1913 claimName: source-storage 1914 - name: experiment-dir 1915 persistentVolumeClaim: 1916 claimName: experiment-storage 1917 params: 1918 - name: application-source 1919 value: https://github.com/kalantar/reviews 1920 - name: application-namespace 1921 value: bookinfo-iter8 1922 - name: application-image 1923 value: kalantar/reviews 1924 - name: application-query 1925 value: productpage 1926 1927 - name: HOST 1928 value: "bookinfo.example.com" 1929 1930 - name: experiment-template 1931 value: iter8/experiment.yaml 1932 </pre> 1933 </div> 1934 </div> 1935 </div> 1936 <div id="outline-container-A%20canary%20%22knative%22%20deployment%20pipeline" class="outline-3"> 1937 <h3 id="A%20canary%20%22knative%22%20deployment%20pipeline"><span class="todo TODO">TODO</span> A canary “knative” deployment pipeline</h3> 1938 </div> 1939 1940 <div id="outline-container-A%20%22matrix%22%20build%20pipeline" class="outline-3"> 1941 <h3 id="A%20%22matrix%22%20build%20pipeline"><span class="todo TODO">TODO</span> A “matrix” build pipeline</h3> 1942 </div> 1943 1944 <div id="outline-container-%3Dtektoncd%2Fpipeline%3D%20project%20pipeline" class="outline-3"> 1945 <h3 id="%3Dtektoncd%2Fpipeline%3D%20project%20pipeline"><span class="todo TODO">TODO</span> <code>tektoncd/pipeline</code> project pipeline</h3> 1946 </div> 1947 1948 <div id="outline-container-Netlify%20flow" class="outline-3"> 1949 <h3 id="Netlify%20flow"><span class="todo TODO">TODO</span> Netlify flow</h3> 1950 <div class="outline-text-3" id="text-Netlify%20flow"> 1951 <ul class="org-ul"> 1952 <li>Build and deploy a wip</li> 1953 </ul> 1954 </div> 1955 </div> 1956 </section> 1957 <section id="outline-container-Issues" class="outline-2"> 1958 <h2 id="Issues"><span class="todo TODO">TODO</span> Issues</h2> 1959 <div class="outline-text-2" id="text-Issues"> 1960 </div> 1961 <div id="outline-container-No%20support%20for%20one-shot%20task%20with%20%3Dgit-clone%3D" class="outline-3"> 1962 <h3 id="No%20support%20for%20one-shot%20task%20with%20%3Dgit-clone%3D">No support for one-shot task with <code>git-clone</code></h3> 1963 <div class="outline-text-3" id="text-No%20support%20for%20one-shot%20task%20with%20%3Dgit-clone%3D"> 1964 <p> 1965 PipelineResource brought <i>pre</i> steps that would help running one task on top of a 1966 GitResource for example. Let’s say you have a repository with a <code>Dockerfile</code>. All you want 1967 is to build your <code>Dockerfile</code> in your CI. Without <code>PipelineResource</code> you are <i>stuck</i> to use a 1968 <code>Pipeline</code>. 1969 </p> 1970 </div> 1971 </div> 1972 </section> 1973 <section id="outline-container-Advantage" class="outline-2"> 1974 <h2 id="Advantage"><span class="todo TODO">TODO</span> Advantage</h2> 1975 </section> 1976 1977 <section id="outline-container-Next%20steps" class="outline-2"> 1978 <h2 id="Next%20steps"><span class="todo TODO">TODO</span> Next steps</h2> 1979 </section> 1980 1981 <section id="outline-container-References" class="outline-2"> 1982 <h2 id="References"><span class="todo TODO">TODO</span> References</h2> 1983 <div class="outline-text-2" id="text-References"> 1984 <ul class="org-ul"> 1985 <li><a href="https://github.com/redhat-gpte-devopsautomation/app-dev-openshift-pipeline">https://github.com/redhat-gpte-devopsautomation/app-dev-openshift-pipeline</a></li> 1986 <li><a href="https://gist.github.com/markito/9ef0329bce51a454e7ce5a0ed18a1eb1">https://gist.github.com/markito/9ef0329bce51a454e7ce5a0ed18a1eb1</a></li> 1987 <li><a href="https://github.com/iter8-tools/canary-tekton-example">https://github.com/iter8-tools/canary-tekton-example</a></li> 1988 <li><a href="https://github.com/ibm/ibm-garage-tekton-tasks">https://github.com/ibm/ibm-garage-tekton-tasks</a></li> 1989 </ul> 1990 </div> 1991 </section> 1992 </main> 1993 <footer id="postamble" class="status"> 1994 <footer> 1995 <small><a href="/" rel="history">Index</a> • <a href="/sitemap.html">Sitemap</a> • <a href="https://dl.sbr.pm/">Files</a></small><br/> 1996 <small class='questions'>Questions, comments ? Please use my <a href="https://lists.sr.ht/~vdemeester/public-inbox">public inbox</a> by sending a plain-text email to <a href="mailto:~vdemeester/public-inbox@lists.sr.ht">~vdemeester/public-inbox@lists.sr.ht</a>.</small><br/> 1997 <small class='copyright'> 1998 Content and design by Vincent Demeester 1999 (<a rel='licence' href='http://creativecommons.org/licenses/by-nc-sa/3.0/'>Some rights reserved</a>) 2000 </small><br /> 2001 </footer> 2002 </footer> 2003 </body> 2004 </html>