laedit | Jérémie Bertrand

Automatically publish a webextension to Firefox, Edge and GitHub

Jérémie Bertrand — Mon, 14 Aug 2023 00:00:00 +0000

I wanted to automatically publish my webextension New Tab - Moment for some time but the lack of possibilities to specify the release notes has stopped me so far.

But there is now a new api on mozilla's addons which provides an endpoint for that, despite it being still not frozen and can change at any time.

So after some thinking, trials and errors I have now the following system:

The release notes are in the Keep a changelog format
A release Firefox action will:
- build the webextension for firefox and edge and outputs two zips, one for each browser
- extract the latest release notes
- sign and publish the firefox zip along with the release notes on AMO
- download the resulting .xpi file
A release GitHub action will create a GitHub release with the release notes, the zips and the xpi files
A release Edge action will upload the edge zip to Microsoft Partner Center Edge dashboard as a draft and submit it for publication

And now for the details.

The following scripts are for GitHub actions but can easily be ported to another system, like the one from Sourcehut for example.
They also presume that the webextension has been created on each browser store hence they only do an update.

I use some external actions in these jobs and since they are still not immutable I reviewed the source code before trusting them with sensitive secrets, but I specify the commit I reviewed in the uses instead of a version to be sure I will use the exact code I have validated.
That said it is always preferable to trust no one with your secrets and if possible write the code that uses them.

Release to Firefox

First the build is done through a shared action, used for a build workflow triggered on pushes and pull requests, and by the release workflow triggered on tags.
It only call the build and package scripts of my package.json then gets the version number from the generated firefox zip to rename the firefox and edge zips and upload them as artifacts:

- run: |
    yarn build
    yarn package
    filename=`ls web-ext-artifacts/firefox/new_tab_-_moment-*.zip | head`
    versionZip=${filename##*-}
    version=${versionZip%.*}
    cp web-ext-artifacts/firefox/new_tab_-_moment-$version.zip web-ext-artifacts/new_tab_-_moment-$version.firefox.zip
    cp web-ext-artifacts/edge/new_tab_-_moment-$version.zip web-ext-artifacts/new_tab_-_moment-$version.edge.zip
  shell: bash

- uses: actions/upload-artifact@v3
  with:
    path: web-ext-artifacts/new_tab_-_moment-*.zip
    if-no-files-found: error

The build script is in charge to:

transpile all typescript files to a build/base folder
copy all other files (html, css, woff2, png) to the same build/base folder
for each browser it will create a specific folder build/{browser}
copy all the files from build/base to each browser folder
then create the specific manifest for each browser since some properties aren't supported by all For the details I leave you to my "to be improved" package.json.

After that the package script calls web-ext build and web-ext lint for each browser folder.

After the build, the release to Firefox action creates the version metadata containing the release notes then signs the webextension on the Mozilla's addons website and finally uploads the resulting .xpi on the build's artefacts:

  - name: Extract release notes
    id: extract-release-notes
    uses: ffurrer2/extract-release-notes@4db7ff8e9cc8a442ab103fd3ddfaebd0f8f36e4c

  - name: Create version metadata
    run: |
        release='${{ steps.extract-release-notes.outputs.release_notes }}'
        cat <<EOF > ./version-metadata.json
        {
          "version": {
            "release_notes": {
              "en-US": $(echo "${release//### }" | jq -sR .)
            }
          }
        }
        EOF

  - run: yarn web-ext sign --api-key ${{ secrets.AMO_ISSUER }} --api-secret ${{ secrets.AMO_SECRET }} --use-submission-api --channel=listed --source-dir build/firefox --amo-metadata ./version-metadata.json

  - uses: actions/upload-artifact@v3
    with:
      path: web-ext-artifacts/new_tab_moment-${{ github.ref_name }}.xpi
      if-no-files-found: error

  outputs:
    release_notes: ${{ steps.extract-release-notes.outputs.release_notes }}

The first two steps focuses on the version metadata: first the release notes of the last version is extracted thanks to ffurrer2's github action then it is inserted in the json file of the version metadata.

Then the web-ext sign command of the well-known webextension tool web-ext to upload the webextension to AMO, sign it and publish it if all went well.
The argument --amo-metadata [metadata file path] (which has to be used with --use-submission-api) allows to specify the version metadata and thus the release notes.

Release to GitHub

The GitHub release runs after the firefox one. It downloads the artifacts uploaded by the previous jobs and creates a new GitHub release with them and the release notes through the softprops/action-gh-release action.

release-github:
  needs: [release-firefox]
  runs-on: ubuntu-latest
  steps:
    - uses: actions/download-artifact@v3

    - name: Create Release
      uses: softprops/action-gh-release@de2c0eb89ae2a093876385947365aca7b0e5f844
      with:
        tag_name: ${{ github.ref }}
        name: ${{ github.ref_name }}
        body: ${{ needs.release-firefox.outputs.release_notes }}
        draft: false
        prerelease: false
        token: ${{ secrets.GITHUB_TOKEN }}
        files: |
          artifact/new_tab_-_moment-${{ github.ref_name }}.*.zip
          artifact/new_tab_moment-${{ github.ref_name }}.xpi

Note that a GitHub access token is needed.

Release to Edge

The Edge release also runs after the firefox one, in parallel of the GitHub release. It downloads the artifacts uploaded by the previous jobs and submit the edge zip as a new version of the edge addon through the wdzeng/edge-addon action.

release-edge:
  needs: [release-firefox]
  runs-on: ubuntu-latest
  steps:
    - uses: actions/download-artifact@v3

    - uses: wdzeng/edge-addon@b1ce0984067e0a0107065e0af237710906d94531
      with:
        product-id: ${{ secrets.EDGE_PRODUCT }}
        zip-path: artifact/new_tab_-_moment-${{ github.ref_name }}.edge.zip
        client-id: ${{ secrets.EDGE_CLIENT }}
        client-secret: ${{ secrets.EDGE_SECRET }}
        access-token-url: ${{ secrets.EDGE_TOKEN_URL }}

And here you go, it is not perfect but largely sufficient for my little addon and maybe yours.

Switching from BlobBackup to rclone

Jérémie Bertrand — Sun, 18 Jun 2023 00:00:00 +0000

In the previous post I used BlobBackup to sync my backup data to a cloud storage and an external drive. Since then the original client hasn't been updated and an account is now mandatory to download and use it. Even if the offer is attractive I didn't want to change my cloud backup location and prefer to know where my data is stored.

So after an extensive search I choose rclone to replace it mainly for the following reasons:

allows access to individual folders and files (even if it means bigger uploads)
well-known open-source software
numerous targets (remotes in rclone language) including those that suit me: S3 - Scaleway and local filesystem
handles encryption of any remote and of the configuration

For the cloud backup I just had to add a new remote to my Scaleway storage named Scaleway, another one to encrypt the first named ScalewayEncrypted and encrypted the configuration.
It is doable in command line but now there is even an experimental gui.

For the external drive backup, a remote configuration wasn't necessary and I choose to encrypt the entire drive and not only the backup data.
I have chosen VeraCrypt for the following reasons:

well-known open-source software
audited
simple to use

Like the backup script I have written a powershell script to ease the sync from and to the cloud and external drive, I will detail it bit by bit:

param(
    [Parameter(Mandatory = $True, Position = 0)]
    [string]
    $syncDirection,
    [Parameter(Mandatory = $True, Position = 1)]
    [string]
    $syncType,
    [Parameter(Mandatory = $False, Position = 2)]
    [string]
    $overrideLocalRoot
)

$ErrorActionPreference = "Stop"
. $PSScriptRoot\Toast.ps1

enum SyncType {
    Cloud
    Local
}

enum SyncDirection {
    To
    From
}

The parameters allows to easily define the sync direction and type, sadly since param must be at the top it can't use the enums directly.
The local root path can be overridden, specially useful for testing a restore and not lose data.

For the popup notification I use BurntToast, here is the Toast.ps1:

enum Icon {
    Info
    Error
}

# Icons from Roselin Christina.S from Noun Project
$icons = @{
    [Icon]::Info = "$PSScriptRoot\info.png";  # https://thenounproject.com/icon/info-1156901/
    [Icon]::Error = "$PSScriptRoot\error.png" # https://thenounproject.com/icon/error-1156903/
}

function Pop-Toast([string] $title, [string] $message, [Icon] $icon)
{
    New-BurntToastNotification -AppLogo $icons[$icon] -Text $title, $message
}

Now the sync configuration part:

class SyncConfiguration {
    [ScriptBlock] $getRemote
    [string] $additionalParameters
}

function Get-DriveLetter([string] $driveName) {
    return (Get-CimInstance -ClassName Win32_Volume | ? { $_.Label?.ToLower() -eq $driveName }).DriveLetter
}

function Get-KasullRemote {
    $kasullLetter = Get-DriveLetter 'kasull'

    while ($null -eq $kasullLetter) {
        Write-Host 'Please insert Kasull, mount it in veracrypt and press a key' -ForegroundColor DarkGreen
        $null = $host.ui.RawUI.ReadKey("NoEcho,IncludeKeyDown")
        $kasullLetter = Get-DriveLetter 'kasull'
    }

    return "$kasullLetter\Backup\"
}

$syncConfigurations = @{
    [SyncType]::Cloud = [SyncConfiguration]@{ getRemote = { "ScalewayEncrypted:rclone/" }; additionalParameters = "--ask-password=false" };
    [SyncType]::Local = [SyncConfiguration]@{ getRemote = { Get-KasullRemote } }
}

class BackupEntry {
    [string] $name
    [string] $localParentPath
}

function Get-BackupEntries() {
    return @(
        [BackupEntry]@{ name = 'Backup'; localParentPath = "D:\" }
        [BackupEntry]@{ name = 'Archives'; localParentPath = "D:\" }
        [BackupEntry]@{ name = 'Photos'; localParentPath = "M:\" }
    )
}

$rclone = "D:\Softwares\rclone\rclone.exe"

Sync configurations are defined for the cloud and local. Since VeraCrypt can mount a drive on any letter, the letter of the external drive is detected with his name.
To fully automate the sync the rclone password is taken from an environment variable hence the --ask-password=false. After that the folders to sync are also defined.

And the last part of the script which determine the rclone arguments and calls it:

try {
    $syncConfiguration = $syncConfigurations[[SyncType]$syncType]

    $title = "Sync $syncDirection $syncType"

    .$rclone selfupdate

    Pop-Toast $title 'Started' Info

    if (![string]::IsNullOrEmpty($overrideLocalRoot) -and !$overrideLocalRoot.EndsWith("\")) {
        $overrideLocalRoot += "\"
    }

    foreach ($backupEntry in Get-BackupEntries) {
        $syncSource = "$([string]::IsNullOrEmpty($overrideLocalRoot) ? $backupEntry.localParentPath : $overrideLocalRoot)$($backupEntry.name)"
        $syncDestination = "$($syncConfiguration.getRemote.invoke())$($backupEntry.name)"

        if ([SyncDirection]$syncDirection -ne [SyncDirection]::To) {
            $destTemp = $syncSource
            $syncSource = $syncDestination
            $syncDestination = $destTemp
        }

        Write-Host "Sync $($backupEntry.name)" -ForegroundColor DarkGreen
        .$rclone sync --progress $syncConfiguration.additionalParameters $syncSource $syncDestination
    }

    Pop-Toast $title 'Finished' Info
}
catch {
    Pop-Toast $title 'Error' Error
    Write-Host 'An error occurred:'
    Write-Host $_.ScriptStackTrace
    Write-Host $_
    Read-Host -Prompt 'Press enter to close'
}

The script can be invoked like this SyncBackup.ps1 To Cloud or SyncBackup.ps1 From Local.

And finally I created two tasks to launch the syncs regularly:

# SyncToCloud
$action = New-ScheduledTaskAction -Execute 'pwsh' -Argument '-File D:\Backup\Scripts\SyncBackup.ps1 To Cloud'
$trigger = New-ScheduledTaskTrigger -Daily -At 2AM
$description = "Sync data to the cloud"
$settings = New-ScheduledTaskSettingsSet -StartWhenAvailable -WakeToRun
Register-ScheduledTask -TaskName "SyncToCloud" -Action $action -Trigger $trigger -Description $description -RunLevel Highest -Settings $settings

# SyncToKasull
$action = New-ScheduledTaskAction -Execute 'pwsh' -Argument '-File D:\Backup\Scripts\SyncBackup.ps1 To Local'
$trigger = New-ScheduledTaskTrigger -Weekly -DaysOfWeek Sunday -At 5AM
$description = "Sync data to Kasull"
$settings = New-ScheduledTaskSettingsSet -StartWhenAvailable -WakeToRun
Register-ScheduledTask -TaskName "SyncToKasull" -Action $action -Trigger $trigger -Description $description -RunLevel Highest -Settings $settings

And that's it! (for now at least)

Switching from Hubic to BlobBackup

Jérémie Bertrand — Sun, 28 Nov 2021 00:00:00 +0000

It's important to do backups.
I had one in place for all my important documents, encrypted locally and synced with Hubic. But since it is becoming out of commission and the last time I tried to do a restore the data was corrupted, I though that it was time to replace it.

Usage

First I needed to rethink my use of backups: I was only doing backup of some important files in a cloud provider.
My plan was to add my photos, some other documents and my software settings to the backup for ~25GB and the backup should be done in cloud and on a local external hard drive.

Settings backup

I have a hard drive with the OS and softwares and another drive with the data to avoid losing them if I need to wipe the OS clean.
The idea was just to copy the settings of certain softwares I use in order to be able to restore them after a fresh install of Windows.
I have written a quick powershell script (I'm tempted to make it a more robust application):

#Requires -RunAsAdministrator
$ErrorActionPreference = "Stop"
Add-Type -AssemblyName System.Windows.Forms
$dataFolder = "D:\Backup\Data"

$balloon = New-Object System.Windows.Forms.NotifyIcon
$balloon.Icon = [System.Drawing.Icon]::ExtractAssociatedIcon((Get-Process -id $pid).Path)

function ToastNotification([string] $message, [System.Windows.Forms.ToolTipIcon] $icon) {
    $balloon.BalloonTipIcon = $icon
    $balloon.BalloonTipText = $message
    $balloon.BalloonTipTitle = $message
    $balloon.Visible = $true
    $balloon.ShowBalloonTip(5000)
}

function CreateBackupDir([string] $dirName) {
    if (-not (Test-Path "$dataFolder\$dirName")) {
        New-Item -ItemType Directory -Force -Path "$dataFolder\$dirName"
    }
}

function DeleteDir([string] $dirName) {
    if (Test-Path "$dataFolder\$dirName") {
        Remove-Item -path "$dataFolder\$dirName" -recurse -force
    }
}

try {
    ToastNotification 'Backup started' Info

    # Jellyfin
    $jellyfinBackupPath = "$dataFolder\JellyfinServer.7z"
    Write-Host 'Backup Jellyfin' -ForegroundColor DarkGreen
    # 1. Stop Jellyfin
    Write-Host 'Stopping service' -ForegroundColor DarkGray
    Stop-Service 'JellyfinServer'
    # 2. Make a copy of all the Jellyfin data and configuration directories
    Write-Host 'Copying data' -ForegroundColor DarkGray
    &'C:\Program Files\7-Zip\7z.exe' a $jellyfinBackupPath 'C:\ProgramData\Jellyfin\Server' -mx=0 -aoa
    # 3. Start Jellyfin
    Write-Host 'Starting service' -ForegroundColor DarkGray
    Start-Service 'JellyfinServer'

    # Firefox dev tabs
    Write-Host 'Backup Firefox' -ForegroundColor DarkGreen
    CreateBackupDir 'Firefox'
    Copy-Item -Path "$env:APPDATA\Mozilla\Firefox\Profiles\*.dev-edition-default\sessionstore-backups\recovery.jsonlz4" -Destination "$dataFolder\Firefox"
    Copy-Item -Path "$env:APPDATA\Mozilla\Firefox\Profiles\*.dev-edition-default\sessionstore-backups\recovery.baklz4" -Destination "$dataFolder\Firefox"

    # Powershell profile
    Write-Host 'Backup Powershell' -ForegroundColor DarkGreen
    if (Test-Path -Path "$env:USERPROFILE\Documents\Powershell")
    {
        Copy-Item -Path "$env:USERPROFILE\Documents\Powershell\Microsoft.PowerShell_profile.ps1" -Destination "$dataFolder\" –Force
    }
    else
    {
        Copy-Item -Path "$env:USERPROFILE\OneDrive\Documents\Powershell\Microsoft.PowerShell_profile.ps1" -Destination "$dataFolder\" –Force
    }

    # BlobBackup
    Write-Host 'Backup BlobBackup' -ForegroundColor DarkGreen
    CreateBackupDir 'BlobBackup'
    Copy-Item -Path "$env:USERPROFILE\.blobbackup\*" -Destination "$dataFolder\BlobBackup\"

    # Notepad++
    Write-Host 'Backup Notepad++' -ForegroundColor DarkGreen
    CreateBackupDir 'Notepad++'
    DeleteDir 'notepad++\backup'
    Copy-Item -Path "$env:AppData\Notepad++\config.xml" -Destination "$dataFolder\Notepad++"
    Copy-Item -Path "$env:AppData\Notepad++\session.xml" -Destination "$dataFolder\Notepad++"
    Copy-Item -Path "$env:AppData\Notepad++\backup" -Destination "$dataFolder\Notepad++" -Recurse

    ToastNotification 'Backup finished' Info
}
catch {
    ToastNotification 'Backup error' Error
    Write-Host 'An error occurred:'
    Write-Host $_.ScriptStackTrace
    Write-Host $_
    Read-Host -Prompt 'Press enter to close'
}

Edit: Thanks to Loïc Wolff the administrator check has been simplified with a #requires and the notifications are now grouped together.
It requires to be launched as administrator to be able to stop and restart the Jellyfin service, it copies the settings of various softwares in a specific folder and raise a notification (Thanks to Boe Prox), alerting me if an error showed.
Edit 2: Since then I discovered that the notifications do not stay in the notification center of Windows 11 so I switched to BurntToast:

enum Icon {
    Info
    Error
}

# Icons from Roselin Christina.S from Noun Project
# https://thenounproject.com/icon/error-1156903/
# https://thenounproject.com/icon/info-1156901/
$icons = @{
    [Icon]::Info = "$PSScriptRoot\info.png";
    [Icon]::Error = "$PSScriptRoot\error.png"
}

function Pop-Toast([string] $title, [string] $message, [Icon] $icon)
{
    New-BurntToastNotification -AppLogo $icons[$icon] -Text $title, $message
}

A scheduled task allows me to run it every day:

$taskAction = New-ScheduledTaskAction -Execute 'pwsh' -Argument '-File Backup.ps1'
$taskTrigger = New-ScheduledTaskTrigger -Daily -At 1AM
$taskName = "Backup"
$description = "Backup settings of applications"

Register-ScheduledTask -TaskName $taskName -Action $taskAction -Trigger $taskTrigger -Description $description -RunLevel Highest

Search for a replacement

Next step was to search for a replacement of Hubic. After a quick search I turned to rclone which is very versatile but seems a little bit complex for my taste.
Luckily a french blogger I follow published an article at that time on a backup software which seems to check all my boxes.
BlobBackup is simple to use, handles many backup destinations including local directory and S3 compatible storages, is open-source and encrypts data.

But there are some inconveniences:

it is still in beta, although very stable
it hasn't an included cloud storage (for now) which forces me to search for one

Cloud storage

My requirements were an S3 compatible storage preferably located in France or europe, and I found Scaleway which is a french cloud provider and has a free Object Storage plan up to 75Go.

I just had to create an account, fill in my credit card to had access to the creation of an S3 bucket and the last thing to do was to create an API key.

BlobBackup Configuration

BlobBackup is very easy to configure, you choose a storage location, fill in the backup name and password and set the storage parameters.
After that you choose the folders to include in the backup, maybe define exclude rules and a schedule and can even specify a retention.

For my needs, I have configured a backup to an external hard drive scheduled every week on sunday 5AM and a backup to my Scaleway object storage bucket every day at 2AM.

Quick feedback

I have only used BlobBackup since ~15 days, but here are my quick feedback.
There are some issues, with storage parameters modification for example, due to his young age but it is open-source and you can help build it.
About his performance, I have ~25 Go of data to backup and thanks to his incremental engine for the backup to the external drive it has taken about ~17 minutes the first time and now it is down to 49 secondes. Of course it depends if many files have been modified.

Switching from Chocolatey to Winget

Jérémie Bertrand — Wed, 24 Nov 2021 00:00:00 +0000

Around every 2-3 years I reinstall my computer with a fresh Windows.
To avoid loosing to much time, I use Boxstarter and Chocolatey to automate as much as possible all settings and softwares installations.
But since I use Chocolatey only on that occasion, I wanted to replace it by Winget: is is (almost) native to windows, the v1.0 is out and the number of packages available in the community repository is good enough.

Both are package managers and are invoked from the command line, so the switch was not hard but there was some differences and deficiencies to come by.

Installing winget

Winget will be integrated with Windows 11 but it is not yet a reality, so to be sure to have the right version installed I use this:

$hasPackageManager = Get-AppPackage -name 'Microsoft.DesktopAppInstaller'
if (!$hasPackageManager -or [version]$hasPackageManager.Version -lt [version]"1.10.0.0") {
    Start-Process ms-appinstaller:?source=https://aka.ms/getwinget
    Read-Host -Prompt "Press enter to continue..."
}

I use the prompt to pause the script since I need winget for the rest.

Package arguments

Chocolatey allows packages creators to add parameters directly for the package, which allows to install Git, add it to the path and disables shell integration with this command:

choco install git.install --params "/GitOnlyOnPath /NoShellIntegration"

Winget doesn't allow package creators to add parameters since it handles directly the installers, but you can override the parameters passed to the installer.
So to install git the same way than above, you can use the --override parameter which looks like this:

winget install --id Git.Git --override '/VERYSILENT /SUPPRESSMSGBOXES /NORESTART /NOCANCEL /SP- /LOG /COMPONENTS="assoc,gitlfs" /o:PathOption=Cmd'

Since you override all parameters, you have to pass the silent ones and the ones you want to add.

It can be hard to find the parameters of the installer, for git you can see in this issue that they have added a /o: arg which overrides the parameters defined in the installer definition, specifically the arguments passed to the ReplayChoice function. For the components, they are defined in the [Components] section.

Here are some other examples:
Visual Studio Code
Choco:

choco install vscode --params "/NoDesktopIcon /NoQuicklaunchIcon"

Winget:

winget install --id Microsoft.VisualStudioCode --override '/VERYSILENT /SUPPRESSMSGBOXES /MERGETASKS="!runcode,!desktopicon,!quicklaunchicon"'

The !runcode task allows to not run Visual Studio Code once installed.

Sumatra PDF
Choco:

choco install sumatrapdf.install --params "/WithPreview"

Winget:

winget install --id SumatraPDF.SumatraPDF --override '/install /S -with-preview'

Missing packages

Some packages where missing from the community repository, but it is quite easy to create a new one with winget create or, if you want to automate that, with the YamlCreate script.
For example I quickly added Cybersoft.DriversCloud and Jellyfin.JellyfinServer.
And if you want the packages to keep up to dates with the releases, you can raise a new issue in the winget-pkgs-automation repository or better, make a new PR to add the required infos and maybe script necessary to automate the updates.

But Winget is still limited to installers like msi, msix and exe so I had to handle the packages without installers otherwise.

For standalone exe I chose to download them but not to add them to the path contrary to Chocolatey:

iwr https://dl5.oo-software.com/files/ooshutup10/OOSU10.exe -out "$env:USERPROFILE\Downloads\OOSU10.exe"

Since it concerns only exe that will be used only once, the add to path is not necessary.
To ease the process I made it a function:

function Download ($url) {
    $fileName =  Split-Path $url -leaf
    $downloadPath = "$env:USERPROFILE\Downloads\$fileName"
    iwr $url -out $downloadPath
    return $downloadPath
}

Same for zip files:

function UnzipFromWeb ($url) {
    $downloadPath = Download $url
    $targetDir = "$env:USERPROFILE\Downloads\$((Get-ChildItem $downloadPath).BaseName)"
    Expand-Archive $downloadPath -DestinationPath $targetDir -Force
    Remove-Item $downloadPath
    return $targetDir
}

Which is used like this:

UnzipFromWeb 'https://github.com/microsoft/Terminal/releases/download/1904.29002/ColorTool.zip'

For font it is a little bit more difficult:

$cascadiaCodeFolder = UnzipFromWeb 'https://github.com/ryanoasis/nerd-fonts/releases/latest/download/CascadiaCode.zip'
$fonts = (New-Object -ComObject Shell.Application).Namespace(0x14)
foreach ($file in "$cascadiaCodeFolder\*.ttf")
{
    $fileName = $file.Name
    dir $file | %{ $fonts.CopyHere($_.fullname) }
}
Remove-Item $cascadiaCodeFolder -Recurse -Force

The code is adapted from Simon Timms blog post.

Boxstarter and Chocolatey uninstall

Since I will not use them after that, I chose to uninstall both Boxstarter and Chocolatey.
It is not easy but the code is available on their website: for boxstarter (at the bottom) and for Chocolatey.
Be careful thought, since it modify the path it can cause quite a damage on your computer.

Tips for creating an ASP.NET Core MVC PWA and hosting it on Ikoula Shared Host

Jérémie Bertrand — Thu, 01 Apr 2021 00:00:00 +0000

Disclaimer: This is not a precise how-to but merely some tips.

Creating an ASP.NET Core MVC PWA

I recently created a personal app and I wanted to try to make a PWA from an ASP.NET Core MVC app.

So once the app done I wanted to add the PWA specific parts which consist of two things:

a manifest which describe the application to browser, in order for them to propose the installation for example
a service worker which is generally used to cache some or all parts of the application in order to allow an offline use

There is a convenient Nuget package WebEssentials.AspNetCore.PWA which appears to do the job well, but I wanted to things myself since it is my first PWA.

Manifest

So I added a manifest.webmanifest in the wwwroot folder, but the uri was returning a 404. I appears that the UseStaticFiles method only returns files for known content type.
So I had to add the right content type with the following code in the Startup.Configure:

var provider = new FileExtensionContentTypeProvider();
provider.Mappings[".webmanifest"] = "application/manifest+json";
app.UseStaticFiles(new StaticFileOptions { ContentTypeProvider = provider});

Service worker

For the service worker, I added his registration through a dedicated script in order to avoid issue with the csp: default-src 'self' policy:

if ('serviceWorker' in navigator) {
    document.addEventListener('DOMContentLoaded', () => {
        try {
            navigator.serviceWorker.register('/serviceworker.js', { scope: '/' });
        } catch (e) {
            console.error('Error during service worker registration', e);
        }
    });
}

It is registered only if the browser supports it and after the dom has loaded to avoid any blocking during the page load.
The log is minimal here, you have to adapt it to your log system.

After that I created the service worker script but was confronted with another issue: I use typescript and by default it doesn't include the webworkers types so I was having errors about unknown types like ExtendableEvent and FetchEvent.
So I found the right StackOverflow answer, reorganized my scripts and used a multi-tsconfig configuration to have all base compiler options common and specialized lib by folder : DOM for the client-side scripts and WebWorker for the service worker.

But I still had errors about the export part:

declare var self: ServiceWorkerGlobalScope;
export {};

It was because for this app I only used scripts and not client-side app, so I had the typescript compiler module option set to none.
So once again after having found the right solution (this time on github) I used the following code:

const sw: ServiceWorkerGlobalScope & typeof globalThis = self as any

and replaced all self by sw.

And after that all was well!

Hosting on a Ikoula shared host

Web deploy

I have a shared host at Ikoula and they use Plesk for that.
I recently discovered that Plesk have an option to activate the web deploy publishing under the hosting settings.
Once activated you have a link on your site's dashboard which allows you to download the web deploy publishing settings.
The format is not the same that the "Import Profile" option expect in Visual Studio 2019, but you can create a new "Web Server (IIS) / Web Deploy" publish profile and copy the values from the file.

If you have an issue on certificate check, that can be resolved through the "Validate Connection" on the Connection page.

SQLite error

I use SQLite with Entity Framework for business and identity storage and after the app deployment I got this error:

DllNotFoundException: Unable to load DLL 'e_sqlite3' or one of its dependencies: Access denied. (0x80070005 (E_ACCESSDENIED))

I don't know how that works, but it appears that the SQLite Entity Framework package need the write permission to the folder where the dll are.
So just add it to the application pool group user and that's done!

Secrets

If you use secrets, it is recommended to use environment variables.
But if you can't (at least I didn't find a way to set one on Plesk), how do you do?

Well I tried to copy the content of the secrets.json in the appsettings.json after the deployment but it is not practical since I use webdeploy and it need to restart the website (a touch of the web.config suffice).

So finally I just used my own secrets.json file, added it to the app configuration and ignored it through the .gitignore to avoid leaking secrets.
It is not the best but at least that do the trick.

So here my few tips, don't hesitate to guide me to better ways!

My journey with NDepend

Jérémie Bertrand — Tue, 16 May 2017 00:00:00 +0000

Disclaimer: I was offered a 1 year NDepend pro license by Patrick Smacchia (NDepend creator) so I could use it and share my experience if I wanted to.
Since I was in the middle of the refactoring of a little personal WPF project, I though that this was the perfect time to test NDepend beyond the testing phase.

The project is a small WPF application which used only code behind, so I decided to refactor it to switch to MVVM and to change the design from WPFSpark to MahApps:

➞

I started using NDepend during the refactoring so it passed some time before all was good, but here is my journey with NDepend.

What is NDepend

I am not good at presenting things so you'd better go to their site, but basically it is a static code analyzer which based on your assemblies and code coverage will bring you a ton a interesting information and metrics about your code.

Installation

The installation is ultra easy: you only have to unzip a folder and launch Visual NDepend to validate the license.
Then you can install the Visual Studio extension or another integration.

An installer and / or a Chocolatey package would be preferable but it does the job.

Integration

NDepend have an extensive list of integrations:

Visual NDepend
NDepend Console
Visual Studio
TFS / VSTS
SonarQube
CruiseControl.NET
FinalBuilder
TeamCity
Jenkins

I only used Visual NDepend, Visual Studio and the VSTS extension.

Visual NDepend

It is the standalone version of NDepend and propose all the functionalities of NDepend as wall as all the installation of Visual Studio extensions, the links to the docs, options, UserVoice and release notes:

It also take care of the updates, which are notified as windows notifications.

Visual Studio

The integration add a new NDepend menu which allow access to all windows / functionalities of NDepend.
There is also a quick access through a round icon in the bottom right:

To create a new NDepend project attached to the current solution it is as simple as click on NDepend Menu / Attach new NDepend project to solution.
Within a couple of seconds my solution of 9 projects (+1 for the WIX installer) have been analyzed and boy, I had work to do!

The integration with Visual Studio is beautiful but could be improved with roslyn integration: having the errors right on the code. It is a topic on NDepend's UserVoice.
Edit: it seems that it will be the next big thing of NDepend and has been announced during the Build event (from 29:00)
Some windows are also oddly placed (the info tooltip appear too low and a part is invisible because it is below the screen).

VSTS

The extension is available on the marketplace and has a free 30 days trial.

It is also easy to configure

I have got some issues to make it work on my VSTS account but the support was responsive and a fix was issued within a day.

Usage

Once the NDepend project properly configured, including the code coverage which needs some extra steps, you can run an analysis if one haven't been launched automatically after a project/solution build.

And now if your project is like mine, you are facing a long list of rules violated, the global status of your debt, a ton of information on your code (types, comments, coverage) and multiple ways to visualize it (dependency graph, dependency matrix, metrics, trends).

So if you want to fix the debt, you just have to browse the rules and either choose to fix the code, or to adapt the rules to your project. You can deactivate some, or add some exceptions.

The good news is that all rules are stored in the .ndproj file, so every rule modification will be applicable for everyone running the project.
And NDepend detect the modification of the .ndproj outside of the application (Visual NDepend or Visual Studio) and propose to reload it automatically. That is excellent but can lead to some freeze of Visual Studio.

One thing I loved is that unlike other static analyzer like Roslyn's analyzers or SonarQube, which I have used and propose only rules related directly to code and best practices, NDepend also provide some architectural level rules (Namespace dependency, project organization, assembly cohesion and such). That forced me to take some steps back and see the "big picture" of my project, which I didn't do in a while.
That have allowed me to realized that most projects on the solution doesn't belong here but only on another. So I have reorganized entirely my solution and focus on the only project that matter.

rules

I will not detail all the rules I have fixed on my project, there is too many and not all have a general interest. But there are some that I think are interesting or can be useful for every WPF / XAML project:

Avoid namespaces mutually dependent

A little refactoring of the designated classes have been sufficient.
One was referencing types that it not needed to know to work, they were just passed to another class which need only object.
The second was using a static property of the App class (in root namespace) to get the log file path, so I moved all log related code to a specialized class.

Methods should be declared static if possible

Cannot be applied to event handler bounded to a XAML window like Loaded. So the rule has been modified with an exception.

Avoid namespaces with few types

My project is small so it is normal that this rule raise a warning, but why is there the XamlGeneratedNamespace in it?
As stated in his name, it is generated so I have no control over it. I think that this rule must ignore namespace which contains only generated types, so I modified the Discard generated Namespaces from JustMyCode which target only the My namespace of VB.NET to also target namespaces which contains only types with GeneratedCode attribute:

notmycode

// First gather assemblies written with VB.NET
let vbnetAssemblies = Application.Assemblies.Where(
   a => a.SourceDecls.Any(decl => decl.SourceFile.FileNameExtension.ToLower() == ".vb"))

// Then find the My namespace and its child namespaces.
let vbnetMyNamespaces = vbnetAssemblies.ChildNamespaces().Where(
   n => n.SimpleName == "My" ||
   n.ParentNamespaces.Any(nParent => nParent.SimpleName == "My"))

let generatedNamespaces = Application.Assemblies.ChildNamespaces().Where(
    n => n.ChildTypes.All(t => t.HasAttribute ("System.CodeDom.Compiler.GeneratedCodeAttribute")))

from n in vbnetMyNamespaces.Concat(generatedNamespaces)
select n

Instance fields naming convention / Fields should be declared as private

I had to specify names for some of my XAML elements and I choose the PascalCase naming convention, so these two rules was failing because the name wasn't corresponding to those of a field and because these XAML fields are generated internal be default.
It is possible to change their modifier through the x:FieldModifier="private" attribute but I considered that since it was part generated (everything but the name) it should not be considered as my code and modified the Discard generated Fields from JustMyCode rule:

notmycode
from f in Application.Fields where
  f.HasAttribute ("System.CodeDom.Compiler.GeneratedCodeAttribute".AllowNoMatch()) ||

  // Eliminate "components" generated in Windows Form Control context
  f.Name == "components" && f.ParentType.DeriveFrom("System.Windows.Forms.Control".AllowNoMatch()) ||
  // Eliminate XAML generated fields
  f.ParentType.Implement("System.Windows.Markup.IComponentConnector")
select f

IComponentConnector is XAML specific (WPF with this namespace, for UWP it is in Window.UI.Xaml.Markup) and is automatically implemented for every Window, Page and UserControl.

Avoid public methods not publicly visible / Methods that could have a lower visibility

All generated methods from properties of my ViewModels are marked but they need to be public for the XAML binding.

Rule modification to exclude all properties from classes which derive from my ViewModelBase:

&& !((m.IsPropertyGetter || m.IsPropertySetter) && m.ParentType.DeriveFrom("QIASI.Client.ViewModels.ViewModelBase"))

Potentially Dead Methods

Since XAML is not analyzed by NDepend, this rule list all properties used in XAML binding only. For now I haven't find a satisfying way to exclude them without avoiding the risk of false negative in the future.
I looked at NDepend.API but it doesn't seems like it is possible to add some information, only to manipulate information provided by NDepend analyzer.

Avoid the Singleton pattern

I also have modified this rule since it target only types with one static field of its parent type, but I wanted to also track the types which use their interface for the static field type.
Here is the modified rule part:

let staticFieldInstances = t.StaticFields.WithFieldTypeIn(t.InterfacesImplemented.Concat(t))
where staticFieldInstances.Count() == 1

Conclusion

Now my project is (almost) all green!

I loved use NDepend and will continue to use it.
I particularly appreciate the flexibility permitted for the rules modification.
It has some flaws, but the gain and possibilities are totally worth it.

Commit to GitHub with Octokit.net

Jérémie Bertrand — Sat, 12 Nov 2016 00:00:00 +0000

GitHub api & Octokit.net

GitHub has an API which amongts many features, can handles commits directly.

Octokit.net is the .net declinaison of octokit, the official GitHub API client.
You can add it to a project through Nuget.

And to use it you just have to instanciate a GitHubClient with a ProductHeaderValue describing your application:

var github = new GitHubClient(new ProductHeaderValue("GithubCommitTest"));

After that you already have access to many operations, like accessing the data of a user:

var user = await github.User.Get("laedit");
Console.WriteLine($"laedit have {user.PublicRepos} public repos");

But in order to commit you have to authenticate yourself, otherwise you can have a "not found" error.

Authentication

There are two ways of doing it, either with login/password or personal access token.
I strongly recommand the personal access token since it has a limited scope and can be revoked easily at any time.

For that:

go to your GitHub's settings/tokens page
clic on "Generate new token"
check at least the "public_repo" scope since it is needed to commit on public repository

Once generated, you can use it in the GitHub client:

github.Credentials = new Credentials("personal_access_token");

The code above is only an example, avoid to store your token directly in source code or in a Version Control System.

One file / one line commit

The API allows to dome some one-line commits for operations on single file:

// github variables
var owner = "laedit";
var repo = "CommitTest";
var branch = "master";

// create file
var createChangeSet = await github.Repository.Content.CreateFile(
                                owner,
                                repo,
                                "path/file.txt",
                                new CreateFileRequest("File creation",
                                                      "Hello World!",
                                                      branch));

// update file
var updateChangeSet = await github.Repository.Content.UpdateFile(
                                owner,
                                repo,
                                "path/file.txt",
                                new UpdateFileRequest("File update",
                                                      "Hello Universe!",
                                                      createChangeSet.Content.Sha,
                                                      branch));

// delete file
await github.Repository.Content.DeleteFile(
                                owner,
                                repo,
                                "path/file.txt",
                                new DeleteFileRequest("File deletion",
                                                      updateChangeSet.Content.Sha,
                                                      branch));

All content is automatically converted to base64, preventing to commit any file other than text, like an image.
This limitation will be removed with PR #1488.

Full commit

But it is also possible to acces the whole Git Data and create a more complex commit step by step. So you have a precise control on the git database but it require more API calls.

For example if you want to add a new commit on top of the last commit if the master branch:

Get the SHA of the latest commit of the master branch

var headMasterRef = "heads/master";
// Get reference of master branch
var masterReference = await github.Git.Reference.Get(owner, repo, headMasterRef);
// Get the laster commit of this branch
var latestCommit = await github.Git.Commit.Get(owner, repo, masterReference.Object.Sha);

Create the blob(s) corresponding to your file(s)

// For image, get image content and convert it to base64
var imgBase64 = Convert.ToBase64String(File.ReadAllBytes("MyImage.jpg"));
// Create image blob
var imgBlob = new NewBlob { Encoding = EncodingType.Base64, Content = (imgBase64) };
var imgBlobRef = await github.Git.Blob.Create(owner, repo, imgBlob);
// Create text blob
var textBlob = new NewBlob { Encoding = EncodingType.Utf8, Content = "Hellow World!" };
var textBlobRef = await github.Git.Blob.Create(owner, repo, textBlob);

Create a new tree with:
- the SHA of the tree of the latest commit as base
- items based on blob(s) or entirelly new

// Create new Tree
var nt = new NewTree { BaseTree = latestCommit.Tree.Sha };
// Add items based on blobs
nt.Tree.Add(new NewTreeItem { Path = "MyImage.jpg", Mode = "100644", Type = TreeType.Blob, Sha = imgBlobRef.Sha });
nt.Tree.Add(new NewTreeItem { Path = "HelloW.txt", Mode = "100644", Type = TreeType.Blob, Sha = textBlobRef.Sha });

// Other way to add a text file directly
// less API call but the content is automatically converted to base64 so only text can be used
var newTreeItem = new NewTreeItem { Mode = "100644", Type = TreeType.Blob, Content = "Hello Universe!", Path = "HelloU.txt" };
nt.Tree.Add(newTreeItem);

var newTree = await github.Git.Tree.Create(owner, repo, nt);

Create the commit with the SHAs of the tree and the reference of master branch

// Create Commit
var newCommit = new NewCommit("Commit test with several files", newTree.Sha, masterReference.Object.Sha);
var commit = await github.Git.Commit.Create(owner, repo, newCommit);

Update the reference of master branch with the SHA of the commit

var headMasterRef = "heads/master";
// Update HEAD with the commit
await github.Git.Reference.Update(owner, repo, headMasterRef, new ReferenceUpdate(commit.Sha));

Once understood it is not quite complex and it allows to learn how Git works with commit creation.

Incremental FTP deploy of a Pretzel site with Creep on AppVeyor

Jérémie Bertrand — Sun, 30 Oct 2016 00:00:00 +0000

In a previous post, I have described how Pretzel could be integrated with AppVeyor in order to generate and deploy a website.

In the deploy part, I was clearing up all the contents on the FTP through a powershell script and then upload everything at every commit.
It can be fine at start, but as the site grows it can be very time consuming and prone to errors.

In order to fix that, I was searching for an incremental deploy tool which handle FTP (since I am limited with that by my hosting) and I came across creep:
it is written in python, can deploy either from a git revision or files hashes and to local file system, FTP or SSH.

In my case I works with a generated site, which is not versioned in git so I will deploy the content of a folder by using files hashes for comparison, to a distant FTP.
The site must be deployed only if the generation went well, and for security I will store the FTP user/password in AppVeyor secure environment variable.

So, first step, add the variables in appveyor.yml, install the necessary softwares and call the powershell script which will be generating and deploying the site:

environment:
  ftp_user:
    secure: replace_with_you_appveyor_encrypted_ftp_user
  ftp_password:
    secure: replace_with_you_appveyor_encrypted_ftp_password

install:
  - choco install pretzel -y
  - ps: $env:Path += ";C:\\Python35;C:\\Python35\\Scripts"
  - pip install creep

cache:
  - '%LOCALAPPDATA%\pip\Cache -> appveyor.yml'

build_script:
- ps: .\BakeAndDeploy.ps1

test: off

artifacts:
- path: src/_site
  name: compiled_site

The PATH is needed to call pip.
The tests are off in order to not waiste time on it.
The artifact part is needed only if you want to keep a backup of the generated site.

And the second part, the powershell script BakeAndDeploy.ps1 itself:

C:\tools\Pretzel\pretzel bake src

if ($lastExitCode -ne 0)
{
    exit -1
}
else
{
    Write-Host "Starting deploy"
    $envConf = '{{""default"": {{""connection"": ""ftp://{0}:{1}@laedit.net""}}}}' -f $env:ftp_user, $env:ftp_password
    creep -e $envConf -d '{""source"": ""hash""}' -b src/_site -y

    if ($lastExitCode -ne 0)
    {
        exit -1
    }
}

The script starts by calling pretzel bake on the site's source, and if the generation went ok it call creep.
That part is quite confuse because it is a JSON string in a powershell script, with mandatory double quotes for each property. You can also use two files (.creep.env and .creep.def) but since I was getting the ftp user/password from environment variable I did not want to have to write two files every time. And sadly creep doesn't support clean command line parameters for that right now.
$envConf is the creep environment configuration, which in this case define a unique connection to my FTP server, with the user/password from environment variables. It is passed through the -e switch.
The creep definition configuration, with the -d switch state that even if the current directory is in a git repository, creep must use the files hashes as comparison method.
The -b switch defines the base directory and -y always answer 'yes' to prompts, allowing a quiet execution.

And now, at each commit only the new or modified files will be deployed and not the entire site.

There is only one remaining minor issues: it is displayed as an error in AppVeyor even if it works. It probably come from the way python writes in console.

NVika 1.0 is out

Jérémie Bertrand — Sat, 01 Oct 2016 00:00:00 +0000

After a while, NVika is finally out of beta.

What does it do

Parse analytics reports and show issues on console or send them to the current build server.

In action:

On AppVeyor:

Hey, but isn't that something SonarQube kinda do?

Yes, and you can even say that SonarQube do much more.

But NVika has some advantages, since it isn't doing the analysis part it is smaller and simpler to use, you only need a command line.

And since it isn't implying a web server (at least for now) it can be used in pull request to enforce the quality standard of your project.

Future

At first I was thinking about a website, to put the consolidated reports on. It would have been accessible to anyone, with repo's settings only accessible by his owner on GitHub, ala CodeCov or other tool integrated to GitHub. But I don't think it is a good idea, since SonarQube already do that and pretty well.

Another idea is to add some integration to other tools: GitHub on pull requests, Slack, Gitter, others?
But since all of them need a user authorization it can't be done only with a command line tool.

What do you think?

integrate sonarqube in a .net project with appveyor

Jérémie Bertrand — Thu, 29 Sep 2016 00:00:00 +0000

What is SonarQube

From their site:
SonarQube is an open platform to manage code quality. As such, it covers the 7 axes of code quality:

Architecture & design
Duplications
Unit tests
Complexity
Potential bugs
Coding rules
Comments

So basically with this you can be pretty sure that your code is good. And since it is open source, you can download it and install a copy on your own server. Or you can use the instance dedicated to open source projects.

Add scan to build

SonarQube works with MSBuild for .net projects through the Scanner for MSBuild, which is available on Chocolatey. There is also an unofficial plugin for F# but it isn't available on the public instance of sonarqube.

All following code examples will be in classic command line and in FAKE, which is a build automation system I use in my projects, like for NVika.

Installation

Two ways to install it, either download it from the SonarQube Scanner for MSBuild page or through Chocolatey:

choco install "msbuild-sonarqube-runner" -y

or with FAKE, which have a lot of helpers:

"msbuild-sonarqube-runner" |> Choco.Install id

Scan

The scan must be started before the build then ended after the build:

Begin:

MSBuild.SonarQube.Runner.exe begin /k:"laedit:Vika" /n:"Vika" /v:"0.1.4" /d:sonar.host.url=https://sonarqube.com /d:sonar.login=[SonarToken]

Or with FAKE:

SonarQube Begin (fun p ->
        {p with
             ToolsPath = "MSBuild.SonarQube.Runner.exe"
             Key = "laedit:Vika"
             Name = "Vika"
             Version = version
             Settings = [ "sonar.host.url=https://sonarqube.com"; "sonar.login=" + environVar "SonarQube_Token" ] })

Mandatory parameters:

/k | Key: key of the project; Must be unique; Allowed characters are: letters, numbers, -, _, . and :, with at least one non-digit.
/n | Name: name of the project; Displayed on the web interface.
/v | Version: version of the project.
/d:sonar.host.url: SonarQube server url; default: http://localhost:9000; must be set to https://sonarqube.com to use the SonarQube public instance.
/d:sonar.login: your login or authentication token. If login is used, you must use the sonar.password with your password as well but this is highly unsecure.

SonarQube_Token is an AppVeyor secure environment variable wich contains the SonarQube token. While not mandatory it is recommended to generate one by project scanned.

There is also a bunch of other parameters available.

Build like usual, with msbuild for example:

"C:\Program Files (x86)\MSBuild\14.0\Bin\MSBuild.exe" /t:Rebuild

Or an helper:

Target "BuildApp" (fun _ ->
    !! "src/NVika/NVika.csproj"
      |> MSBuildRelease buildResultDir "Build"
      |> Log "AppBuild-Output: "
)

Then the end part:

MSBuild.SonarQube.Runner.exe end /d:sonar.login=[SonarToken]

or with FAKE:

SonarQube End (fun p ->
        {p with
             ToolsPath = "MSBuild.SonarQube.Runner.exe"
             Settings = [ "sonar.login=" + environVar "SonarQube_Token" ]
        })

Since all security related parameters aren't write to the disk, you have to pass them again to the end part. Meaning that if you have used login + password in begin you have to pass them both again in end.

Note: there is no need to create a project in the web interface, it is automatically created on the first analysis.

Frequency

It's up to you to determine the frequency of the SonarQube scans, but int order to avoid abuse of the SonarQube public instance and because a scan is not needed for every local build I choose to start one only on AppVeyor and for the original repository (not forks) because I use AppVeyor's secure environment variables:

// check if the build is on AppVeyor and for the original repository
let isOriginalRepo = environVar "APPVEYOR_REPO_NAME" = "laedit/vika"
let isAppVeyorBuild = buildServer = AppVeyor

// build dependencies: SonarQube scan will be launched only if the condition is true
"Clean"
  ==> "RestorePackages"
  =?> ("BeginSonarQube", isAppVeyorBuild && isOriginalRepo)
  ==> "BuildApp"
  =?> ("EndSonarQube", isAppVeyorBuild && isOriginalRepo)

Additional settings

You might want to subscribe to notifications in order to be aware of each new issues or changes of the quality gate status.

But if you use the public instance, subscribe only for specific projects if you want to avoid getting spammed with notifications of all projects.

Also, even if SonarQube doesn't propose built-in badges, shields do, so you can add one to your project's ReadMe.

Use SonarQube in Pull Request builds

SonarQube have a GitHub, which can handle the pull request build without pushing the results to SonarQube.

You just have to add several parameters:

sonar.analysis.mode=preview: avoid to send the results to the SonarQube instance
sonar.github.pullRequest=" + environVar "APPVEYOR_PULL_REQUEST_NUMBER": pull request number, here from AppVeyor
sonar.github.repository=laedit/vika: identification of the repository with format <organisation/repo>
sonar.github.oauth=" + environVar "Sonar_PR_Token": GitHub personal access token, with the scopes public_repo (or repo for private repositories) and repo:status in order to update the PR status

In this example, Sonar_PR_Token is the GitHub token embedded as AppVeyor secure environment variable. They are not accessible from PRs, unless you check the "Enable secure variables in Pull Requests from the same repository only" box in General tab of your AppVeyor's repo settings.

Even if only PRs from the same repository will have access to the content of the secure environment variable, they can still be visible in the logs of your AppVeyor builds, so be careful.

But if you still want to implement it, be sure to add these parameters only on PRs, for example with FAKE:

let isPR = environVar "APPVEYOR_PULL_REQUEST_NUMBER" |> isNull |> not

Target "BeginSonarQube" (fun _ ->
    "msbuild-sonarqube-runner" |> Choco.Install id

let sonarSettings = match isPR with
                        | false -> [ "sonar.host.url=https://sonarqube.com"; "sonar.login=" + environVar "SonarQube_Token" ]
                        | true -> [
                                    "sonar.host.url=https://sonarqube.com";
                                    "sonar.login=" + environVar "SonarQube_Token";
                                    "sonar.analysis.mode=preview";
                                    "sonar.github.pullRequest=" + environVar "APPVEYOR_PULL_REQUEST_NUMBER";
                                    "sonar.github.repository=laedit/vika";
                                    "sonar.github.oauth=" + environVar "Sonar_PR_Token"
                                  ]

    SonarQube Begin (fun p ->
        {p with
             ToolsPath = "MSBuild.SonarQube.Runner.exe"
             Key = "laedit:Vika"
             Name = "Vika"
             Version = version
             Settings = sonarSettings })
)

SonarLint

If you want to find issues before committing, you can use SonarLint, either in your favorite IDE or through command line.

migrating the old reader bookmark from addon sdk to webextensions

Jérémie Bertrand — Mon, 27 Jun 2016 00:00:00 +0000

Here is the comparison of the commits pushed for the addon's migration.

The Addon

The Old Reader - Bookmark is an addon wich add a button to easily add a page or a selection of the page to The Old Reader bookmarks.
It is done with Firefox Addon SDK, both high-level and low-level APIs.

Preparation

First thing to do: read some docs on WebExtensions.

And a good thing to have is web-ext (available on GitHub), which is a command line tool aiming to help running and debugging WebExtensions.
It is available through npm:

npm install --global web-ext

In order to ease the process, I have created a small cmd file which will update web-ext if necessary and run it :

@echo off
:: update web-ext
call npm update -g web-ext

echo Exit Code is %errorlevel%
if "%ERRORLEVEL%" == "1" exit /B 1

:: run web-ext
if [%1]==[] (
    web-ext -s "src" run --firefox-binary "C:\Program Files\Firefox Developer Edition\firefox.exe"
)

if [%1]==[current] (
    web-ext -s "src" run --firefox-binary "C:\Program Files\Mozilla Firefox\firefox.exe"
)

if [%1]==[beta] (
    web-ext -s "src" run --firefox-binary "C:\Program Files (x86)\Mozilla Firefox Beta\firefox.exe"
)

web-ext is run against the src folder which contains the WebExtension source and launch Firefox Developer Edition by default but can launch the current or beta version with the right argument.

Migration

For the detail, lets begin with the folder tree before:

|- data
|  |- oldreadericon-16.png
|  |- oldreadericon-32.png
|  +- oldreadericon-64.png
|
|- lib
|  +- main.js
|
|- locale
|  |- en-US.properties
|  +- fr-FR.properties
|
|- icon.png
|- icon64.png
+- package.json

And after:

|- _locales
|  |- en
|  |  +- messages.json (moved and migrated from en-US.properties)
|  +- fr
|     +- messages.json (moved and migrated from fr-FR.properties)
|
|- content_scripts
|  |- getSelection.js
|  +- postNewBookmark.js
|
|- icons (renamed from data)
|  |- oldreadericon-16.png
|  |- oldreadericon-32.png
|  |- oldreadericon-48.png (moved from icon.png)
|  +- oldreadericon-64.png
|
|- background-script.cs (moved and migrated from main.js)
+- manifest.json (migrated from package.json)

Nothing complicated, just some files moved except for three parts:

localization
manifest
logic

localization

That was the easiest part:

rename folder from locale to _locales
create a subfolder fo each language (instead of the culture previously used)
migrate each file from properties to the new format in json

Each entry must have a key which will be used in the extension and a message property which contains the translation. The description property isn't mandatory but could be useful.

{
  "theOldReaderSelfBookmarkMessage": {
    "message": "If you were to bookmark The Old Reader with The Old Reader then the universe will fold in on itself and become a very large black hole.",
    "description": "Message when the user want to bookmark the old reader itself"
  },
  "addonDescription": {
    "message": "Bookmark the current page or selection in The Old Reader (premium membership needed)",
    "description": "addon description"
  }
}

manifest

The manifest stay in json but the format change to be close that the one used by Chrome.
As you can see by comparing the old package.json with the new manifest.json, some properties are quite the same:

the title is now the name
the version doesn't move
the description can use a translated message, you have to use the following pattern: __MSG_messageKey__

Some are new:

the key manifest_version is mandatory, with the value 2 for now
the default_locale key is also mandatory if you have a _locales folder
all icons are now defined in the manifest under the icons key
the applications key with the gecko subkey is specific to firefox, it is used in my addon to define:
- the minimal version of firefox supported
- the addon id - you can see that I have suffixed it with @jetpack: the id must now contains a @, but the jetpack part could be replaced by anything at your like
the persmissions allow to define the permissions needed by the addon. In my case I need the activeTab to get the url of the current tab, and <all_urls> to execute my addon on any website.
the browser_action defines a button on the browser's toolbar, with an icon and a title
the background defines the background scripts and page which are loaded at the launch of the extension, generally they contains the logic of the extension

And some have disappear:

I haven't find where to put the license information
same for author

logic

The logic of the extension take place in the background script, but since WebExtensions support e10s all interactions with the IHM or current page must be injected through a content script, even an alert(...) for showing a small information.

So, there is 4 parts in my old logic script:

toolbar button declaration, wich have been moved to the manifest.json
show an alert if the current tab is on theoldreader.com
get the selection of the current tab if any
post the selection and the url of the current tab to the old reader in a new tab

For the alert I had to use the tabs.executeScript method which allow to inject a code or a script in a tab:

if(/^http(?:s?)\:\/\/theoldreader\.com/i.test(tab.url))
{
    browser.tabs.executeScript({ code: "alert('" + chrome.i18n.getMessage("theOldReaderSelfBookmarkMessage") + "');" });
    return false;
}

You can also see the use of i18n.getMessage to get a translated message from the locales.

After that, I inject the content of getSelection.js:

browser.tabs.executeScript({ file: "content_scripts/getSelection.js" }, postToOldReadBookmarks);

Which get the selection of the current tab and return it. The second parameter is a callback method which handle the return of the script execution, in my extension it is the 4th part which post the data to the old reader in a new tab.

function postToOldReadBookmarks(selections) {
    browser.tabs.create({ index: currentTabIndex + 1, url: "https://theoldreader.com/bookmarks/bookmark" }, function (tab) {
        browser.tabs.executeScript(tab.id, { file: "content_scripts/postNewBookmark.js" }, function () {
            chrome.tabs.sendMessage(tab.id, {url: currentTabUrl, html: selections});
        });
    });
}

With the Addon SDK it was possible to post directly the data to a new tab but with WebExtensions it is not (yet?) possible, instead I use a form created in a content script.
Due to some limitations and bugs, I must inject the script to a 'real' page, not a about:blank page or one included in my extension.
After that I use the tabs.sendMessage to pass the data to the script, which will inject it as hidden input value before submitting the form.

And all works!

conclusion

Right now WebExtensions are still in development and even if Firefox 48 have brought the first stable release I prefer to wait until at least v49 (which fix some issues I have encontered) is out before publishing the update of my addon.
Nevertheless WebExtensions are very promising and the shared APIs with chrome, opera and even edge is a big asset, I will have to test my extension on those!

integrate pretzel with appveyor

Jérémie Bertrand — Tue, 14 Jun 2016 00:00:00 +0000

Pretzel is a static web site generator, much like Jekyll but in .net.
So while Jekyll users can use TravisCI or directly GitHub pages to generate their website we don't have this possibility with Pretzel.

Luckily, Appveyor provides the same features than TravisCI but in a Windows environment. And it is free, so you just have to create an account or login with your GitHub account.

Like travis, it is based on a yaml configuration file, named appveyor.yml, like the one for my website.

Preparation

First, we need to install pretzel:

install:
  - choco install pretzel -y

cache:
  - C:\tools\Pretzel -> appveyor.yml

The cache instruction indicates appveyor to store in cache the content of the C:\tools\Pretzel folder until the appveyor.yml is modified.

The installation folder of pretzel will soon be modified to comply to Chocolatey rules.

Build

The build_script is simple: just run the bake command of pretzel on the site source.

build_script:
- C:\tools\Pretzel\pretzel bake src

artifacts:
- path: src/_site
  name: compiled_site

Since the pretzel folder is in the appveyor cache, we cannot use the pretzel exe from the PATH, we must use the full path.

The artifacts instruction defines the files or folder you want to save after a build. You can access and download these files directly on the appveyor website.

Test

test: off

So I think this line is self-explanatory: no test at all. For now at least, I plan to add a link checker soon. And maybe some sort of integration tests, we'll see.

Deploy

Since this post I have set an incremental FTP deploy with creep.

So, now the goal is to deploy the artifact in a FTP server.
You can also deploy it to GitHub, Azure and other, appveyor support quite a few deployment supports.

before_deploy:
- ps: .\Clear-FtpDirectory.ps1

deploy:
- provider: FTP
  host: laedit.net
  protocol: ftp
  username: zlaeditn12713ne
  password:
    secure: eK/zCvZEGU6BcRfo1CoYnlrLD7SoyaUyOb3aIq8CkmQ=
  folder: /httpdocs/laedit
  application: src\compiled_site.zip

The before_deploy instruction run a powershell (indicated by the ps: prefix) which cleans the destination folder.
I use powershell but you can use a Fake or your favorite build helper if you prefer.

Then the deploy instruction list all the deploy informations:

provider type
FTP host
protocol used
username used
password encrypted by appveyor and only decrypted during build (and not accessible during PR build +1 for security)
destination folder
the artifact to deploy

And since there is no constraints to the deploy, it is executed at each commit.

Conclusion

You now know how to use pretzel and appveyor to build and publish your shiny static website
You are just a commit away to your next post.

hi

Jérémie Bertrand — Fri, 27 May 2016 00:00:00 +0000

Hi,

This is my personal blog.
I wanted to make a multilingual blog with Pretzel but right now the project need some modification to to that.
So basically this is still a WIP for now...
Meanwhile I will be discovering the limits and possibilities of Pretzel, and integrate it with AppVeyor.

There is no comments for now but you can contact me through mastodon or mail.

What is Pretzel

Pretzel is a static website generator, like Jekyll but in .net.

What is AppVeyor

AppVeyor is a Continuous Integration and Deployment service backed by Windows Azure, like Travis but for Windows environments.

laedit | Jérémie Bertrand

Automatically publish a webextension to Firefox, Edge and GitHub

Release to Firefox

Release to GitHub

Release to Edge

Switching from BlobBackup to rclone

Switching from Hubic to BlobBackup

Usage

Settings backup

Search for a replacement

Cloud storage

BlobBackup Configuration

Quick feedback

Switching from Chocolatey to Winget

Installing winget

Package arguments

Missing packages

Boxstarter and Chocolatey uninstall

Tips for creating an ASP.NET Core MVC PWA and hosting it on Ikoula Shared Host

Creating an ASP.NET Core MVC PWA

Manifest

Service worker

Hosting on a Ikoula shared host

Web deploy

SQLite error

Secrets

My journey with NDepend

What is NDepend

Installation

Integration

Visual NDepend

Visual Studio

VSTS

Usage

rules

Avoid namespaces mutually dependent

Methods should be declared static if possible

Avoid namespaces with few types

Instance fields naming convention / Fields should be declared as private

Avoid public methods not publicly visible / Methods that could have a lower visibility

Potentially Dead Methods

Avoid the Singleton pattern

Conclusion

Commit to GitHub with Octokit.net

GitHub api & Octokit.net

Authentication

One file / one line commit

Full commit

Incremental FTP deploy of a Pretzel site with Creep on AppVeyor

NVika 1.0 is out

What does it do

Links

Future

integrate sonarqube in a .net project with appveyor

What is SonarQube

Add scan to build

Installation

Scan

Frequency

Additional settings

Use SonarQube in Pull Request builds

SonarLint

migrating the old reader bookmark from addon sdk to webextensions

The Addon

Preparation

Migration

localization

manifest

logic

conclusion

integrate pretzel with appveyor

Preparation

Build

Test

Deploy

Conclusion

hi

What is Pretzel

What is AppVeyor