// -------------------------------------------------------------------------------- //

The following steps are for workstations/servers with the SEMS environment installed.

// -------------------------------------------------------------------------------- //
Summary:

- Step 1: Rigorous testing of Kokkos' develop branch for each backend (Serial, OpenMP, Threads, Cuda) with all supported compilers.

- Step 2: Snapshot Kokkos' develop branch into current Trilinos develop branch.

- Step 3: Build and test Trilinos with combinations of compilers, types, backends.

- Step 4: Promote Kokkos develop branch to master if the snapshot does not cause any new tests to fail; else track/fix causes of new failures.

- Step 5: Snapshot Kokkos tagged master branch into Trilinos and push Trilinos.
// -------------------------------------------------------------------------------- //


// -------------------------------------------------------------------------------- //

Step 1:
  1.1. Update kokkos develop branch (NOT a fork)

         (From kokkos directory):
         git fetch --all
         git checkout develop
         git reset --hard origin/develop

  1.2. Create a testing directory - here the directory is created within the kokkos directory

         mkdir testing
         cd testing

  1.3. Run the test_all_sandia script; various compiler and build-list options can be specified

         ../config/test_all_sandia

  1.4 Clean repository of untracked files

        cd ../
        git clean -df

// -------------------------------------------------------------------------------- //

Step 2:
  2.1 Update Trilinos develop branch

        (From Trilinos directory):
        git checkout develop
        git fetch --all
        git reset --hard origin/develop
        git clean -df

  2.2 Snapshot Kokkos into Trilinos - this requires python/2.7.9 and that both Trilinos and Kokkos be clean - no untracked or modified files

        module load python/2.7.9
        python KOKKOS_PATH/config/snapshot.py KOKKOS_PATH TRILINOS_PATH/packages

// -------------------------------------------------------------------------------- //

Step 3:
  3.1. Build and test Trilinos with 4 different configurations; Run scripts for white and shepard are provided in kokkos/config/trilinos-integration

         Usually its a good idea to run those script via nohup. 
         You can run all four at the same time, use separate directories for each.

  3.2. Compare the failed test output between the pristine and the updated runs; investigate and fix problems if new tests fail after the Kokkos snapshot

// -------------------------------------------------------------------------------- //

Step 4: Once all Trilinos tests pass promote Kokkos develop branch to master on Github
  4.1. Generate Changelog (You need a github API token)
    
       Close all Open issues with "InDevelop" tag on github
   
       (Not from kokkos directory)
       gitthub_changelog_generator kokkos/kokkos --token TOKEN --no-pull-requests --include-labels 'InDevelop' --enhancement-labels 'enhancement,Feature Request' --future-release 'NEWTAG' --between-tags 'NEWTAG,OLDTAG'
       
       (Copy the new section from the generated CHANGELOG.md to the kokkos/CHANGELOG.md)
       (Make desired changes to CHANGELOG.md to enhance clarity)
       (Commit and push the CHANGELOG to develop)

  4.2 Merge develop into Master
              
       - DO NOT fast-forward the merge!!!!

       (From kokkos directory):
       git checkout master
       git fetch --all
       # Ensure we are on the current origin/master
       git reset --hard origin/master
       git merge --no-ff origin/develop

  4.3. Update the tag in kokkos/config/master_history.txt
       Tag description: MajorNumber.MinorNumber.WeeksSinceMinorNumberUpdate
       Tag format: #.#.##

       # Prepend master_history.txt with 
       
       # tag: #.#.##
       # date: mm/dd/yyyy
       # master: sha1
       # develop: sha1
       # -----------------------

       git commit --amend -a

       git tag -a #.#.##
         tag: #.#.##
         date: mm/dd/yyyy
         master: sha1
         develop: sha1

   4.4. Do NOT push yet

// -------------------------------------------------------------------------------- //

Step 5:
  5.1. Make sure Trilinos is up-to-date - chances are other changes have been committed since the integration testing process began. If a substantial change has occurred that may be affected by the snapshot the testing procedure may need to be repeated

       (From Trilinos directory):
       git checkout develop
       git fetch --all
       git reset --hard origin/develop
       git clean -df

  5.2. Snapshot Kokkos master branch into Trilinos

       (From kokkos directory):
       git fetch --all
       git checkout tags/#.#.##
       git clean -df

       python KOKKOS_PATH/config/snapshot.py KOKKOS_PATH TRILINOS_PATH/packages
       
  5.3. Run checkin-test to push to trilinos using the CI build modules (gcc/4.9.3)

       The modules are listed in kokkos/config/trilinos-integration/checkin-test
       Run checkin-test, forward dependencies and optional dependencies must be enabled
       If push failed because someone else clearly broke something, push manually. 
       If push failed for unclear reasons, investigate, fix, and potentially start over from step 2 after reseting your local kokkos/master branch

Step 6: Push Kokkos to master
      
       git push --follow-tags origin master 

// -------------------------------------------------------------------------------- //