So, yes this is selfhosting related. I am working on an n8n flow to pull in weather data so that I can have this data on a dashboard. I can’t find any dockerized weather forecasting apps. Most of them connect to a personal weather station, which might be an option in the future. For the time being, this is a little project I’m working on.

Partial JSON snippet:

spoiler 
0	
json	
cod	"200"
message	0
cnt	40
list	
0	
dt	1780693200
main	
temp	29.4
feels_like	29.23
temp_min	29.4
temp_max	29.68
pressure	1019
sea_level	1019
grnd_level	984
humidity	42
temp_kf	-0.28
weather

I would like to display something like this:

spoiler 
Current temperature: 23.25 °C
Feels like: 24.09 °C
Low / High: 23.25 °C / 23.60 °C
Humidity: 94%
Atmospheric pressure: 1023 hPa (sea level: 1023 hPa, ground level: 988 hPa)
Temperature correction factor (temp_kf): -0.35

Weather: Light rain
Weather code: 500
Short condition: Rain
Icon: 10d

So, this is for you devs or coders out there. I can produce the JSON data. I’m just not sure how to parse it to something meaningful. I’m sure Python will have to be incorporated, but unsure of how to proceed.

Maybe someone could point me in a direction to tuts, articles, or your own experience. Sorry the JSON data doesn’t format correctly. Lemmy formatting doesn’t seem to allow that.

ETA: For anyone wandering across this, here is the solution: https://lemmy.world/post/47961985/24201369 thanks to @captcha_incorrect for his generosity and time.

  • chrash0@lemmy.worldEnglish
    5·
    2 months ago

    i use Nushell for this! works with JSON, YAML, TOML, markdown, Polars Dataframes, SQLite, and a bunch of others including builtin parsing tools for whatever formats and a plugin ecosystem. i use it at work and for personal projects as my main shell, and it’s super handy for exploring, unpacking, sorting, and visualizing all sorts of data. i use it to:

    • find specific parts of YAML cloud configs
    • visualize JSON logs, including a parser that restructures journalctl logs.
    • _re_structure data from CLIs to work with them as structured: git logs, Unix coreutils, etc
    • script my environment: common kubectl queries, specific web API helpers, building and running and testing applications, etc

    it is a slight learning curve, and technically you could do all of that with bash or zsh and jq or jc, but i appreciate the modern take on your base shell terminal env.

    it’s replaced both Python and Bash for me.

  • GreenKnight23@lemmy.worldEnglish
    3·
    2 months ago

    lots of ways to do it, where will it be displayed? if it’s in a console, jq would more than suffice.

    if you want it to be processed heavily and added to an HTML document, probably something like a mix of jq and envsubst. you could forego all that and just use sed to inject the json into HTML then have JavaScript parse it out for you browser side.

    if you want to store it in a database why not just store the whole json?

    your requirements aren’t very clear on what you want this data for or how it will be used. without that there’s just too many variable solutions.

    can you parse json? sure! what for/how will it be used? not sure.

    hope this helps.

    • irmadlad@lemmy.worldOPEnglish
      2·
      2 months ago

      envsubst sed

      Ahh more to explore. Thanks.

      your requirements aren’t very clear on what you want this data for or how it will be used.

      Apologies. As per usual, it’s all in my head and sometimes doesn’t make it to paper. I am using Homarr dashboard. Homarr has the ability to incorporate iframes. Ideally I would like to parse the JSON data into a prettier format perhaps adding some icons in the future once I get it all hammered out, and display it in an iframe on my dashboard.

      Thank you for the leads and your time.

      • GreenKnight23@lemmy.worldEnglish
        2·
        2 months ago

        I don’t know much about homarr but it sounds like going with jq and envsubst might give you a faster solution that you can just iframe in your dashboard. though the learning curve can be steep if you’re unfamiliar with Linux command line.

        I’m pretty sure you can literally pass in base64 encoded HTML to an iframe it will load in or pass it a publicly accessible file and it will work too.

        I had messed around with building a locally hosted dashboard using similar methods so I’m pretty sure it will work.

  • captcha_incorrect@lemmy.worldEnglish
    2·
    2 months ago

    Since you are using n8n, pull the data on a schedule and save it in a data table. Then create a webhook to serve the content as you want it. N8n can transform the JSON data, I can provide a sample tomorrow.

    • irmadlad@lemmy.worldOPEnglish
      1·
      2 months ago

      I can provide a sample tomorrow.

      That would be swell, but I wouldn’t make it a priority. Just whenever you get a round to it. I have been given a ton of source material to read. Stayed up a little too late last night trying to assimilate it all. LOL

      • captcha_incorrect@lemmy.worldEnglish
        2·
        2 months ago

        Sharing a workflow requires a paid plan but could should be able to paste this in the editor to get the same effect.

        I made two different sample flows, on that call the api and responds with directly with the response data (transformed). The other workflow is to same the data in a data table, and call the weather api on a schedule. You could even use ntfy and send notifications on a schedule if it will rain within an hour.

        Edit the nodes to suite your need (URL, content of response etc).

        n8n sample workflow (copy and paste in n8n's editor) 
        {
  "nodes": [
    {
      "parameters": {
        "url": "https://opendata-download-metfcst.smhi.se/api/category/snow1g/version/1/geotype/point/lon/16.158/lat/58.5812/data.json",
        "options": {}
      },
      "type": "n8n-nodes-base.httpRequest",
      "typeVersion": 4.4,
      "position": [
        656,
        336
      ],
      "id": "d47a17c9-9bc7-448c-841a-f3b2d787b2cb",
      "name": "Sample weather JSON data"
    },
    {
      "parameters": {
        "jsCode": "let output = [];\n\nfor (const item of $input.first().json.timeSeries) {\n  output.push({\n    Time: item.time,\n    \"Current temperature\": item.data.air_temperature + ' °C'\n    // Add more as needed\n  });\n}\n\nreturn output;\n\n"
      },
      "type": "n8n-nodes-base.code",
      "typeVersion": 2,
      "position": [
        880,
        240
      ],
      "id": "cf4f11a6-0ecc-4e96-afcb-a7903113da19",
      "name": "Convert json to array"
    },
    {
      "parameters": {
        "mode": "raw",
        "jsonOutput": "={\n  \"Time\": {{ $json.timeSeries[0].time }},\n  \"Current temperature\": {{ $json.timeSeries[0].data.air_temperature }},\n}\n",
        "options": {}
      },
      "type": "n8n-nodes-base.set",
      "typeVersion": 3.4,
      "position": [
        880,
        432
      ],
      "id": "2f3765ba-6ee5-44b7-ac45-c5a407ed43e9",
      "name": "Manual mapping of one item",
      "disabled": true
    },
    {
      "parameters": {
        "options": {}
      },
      "type": "n8n-nodes-base.respondToWebhook",
      "typeVersion": 1.5,
      "position": [
        1104,
        240
      ],
      "id": "badd76a3-2920-4592-baa6-5202b4badba0",
      "name": "Respond to Webhook"
    },
    {
      "parameters": {
        "operation": "upsert",
        "dataTableId": {
          "__rl": true,
          "mode": "list",
          "value": ""
        },
        "columns": {
          "mappingMode": "defineBelow",
          "value": {},
          "matchingColumns": [],
          "schema": [],
          "attemptToConvertTypes": false,
          "convertFieldsToString": false
        },
        "options": {}
      },
      "type": "n8n-nodes-base.dataTable",
      "typeVersion": 1.1,
      "position": [
        1104,
        16
      ],
      "id": "c76d9dfa-9ec1-421f-a96b-d1a65efb0dd7",
      "name": "insert (or update if exists) table with data"
    },
    {
      "parameters": {
        "content": "Create the table first, then edit and select how to map the fields\n\n",
        "height": 224,
        "width": 320
      },
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        992,
        -48
      ],
      "typeVersion": 1,
      "id": "51347b11-fcb8-49db-bd17-405ec8719ec0",
      "name": "Sticky Note"
    },
    {
      "parameters": {
        "jsCode": "let output = [];\n\nfor (const item of $input.first().json.timeSeries) {\n  output.push({\n    Time: item.time,\n    \"Current temperature\": item.data.air_temperature + ' °C'\n    // Add more as needed\n  });\n}\n\nreturn output;\n\n"
      },
      "type": "n8n-nodes-base.code",
      "typeVersion": 2,
      "position": [
        880,
        16
      ],
      "id": "3967c4c5-5443-40d2-b1fe-830aa9327118",
      "name": "Convert json to array1"
    },
    {
      "parameters": {
        "url": "https://opendata-download-metfcst.smhi.se/api/category/snow1g/version/1/geotype/point/lon/16.158/lat/58.5812/data.json",
        "options": {}
      },
      "type": "n8n-nodes-base.httpRequest",
      "typeVersion": 4.4,
      "position": [
        656,
        16
      ],
      "id": "f0043f80-c940-43cf-a5ef-2f7fe6dcc6a7",
      "name": "Sample weather JSON data1"
    },
    {
      "parameters": {
        "path": "1572b231-edc5-4516-935c-f205b737c5d2",
        "responseMode": "responseNode",
        "options": {}
      },
      "type": "n8n-nodes-base.webhook",
      "typeVersion": 2.1,
      "position": [
        432,
        336
      ],
      "id": "30953c4e-7627-4646-8b8d-ca46cc06c841",
      "name": "Webhook - Respond after API call",
      "webhookId": "1572b231-edc5-4516-935c-f205b737c5d2"
    },
    {
      "parameters": {
        "rule": {
          "interval": [
            {
              "field": "minutes"
            }
          ]
        }
      },
      "type": "n8n-nodes-base.scheduleTrigger",
      "typeVersion": 1.3,
      "position": [
        448,
        16
      ],
      "id": "e242dea7-dc3c-4c0a-9630-786bf2a39b8a",
      "name": "Schedule to get weather data"
    },
    {
      "parameters": {
        "path": "1572b231-edc5-4516-935c-f205b737c5d2",
        "responseMode": "responseNode",
        "options": {}
      },
      "type": "n8n-nodes-base.webhook",
      "typeVersion": 2.1,
      "position": [
        448,
        -240
      ],
      "id": "1c1ac8d1-5f64-4bab-a3ac-20f223b1411e",
      "name": "Webhook - respond with table data",
      "webhookId": "1572b231-edc5-4516-935c-f205b737c5d2"
    },
    {
      "parameters": {
        "operation": "get",
        "dataTableId": {
          "__rl": true,
          "mode": "list",
          "value": ""
        }
      },
      "type": "n8n-nodes-base.dataTable",
      "typeVersion": 1.1,
      "position": [
        656,
        -240
      ],
      "id": "7433815b-229c-4ad8-88ce-2bcff688ece5",
      "name": "Get row(s)"
    },
    {
      "parameters": {
        "options": {}
      },
      "type": "n8n-nodes-base.set",
      "typeVersion": 3.4,
      "position": [
        864,
        -240
      ],
      "id": "76cd6bfe-da95-41c0-a3ee-812956eab79f",
      "name": "Edit Fields"
    },
    {
      "parameters": {
        "options": {}
      },
      "type": "n8n-nodes-base.respondToWebhook",
      "typeVersion": 1.5,
      "position": [
        1072,
        -240
      ],
      "id": "3c83ebc3-2c09-4d80-8278-e325237ae0e2",
      "name": "Respond to Webhook1"
    },
    {
      "parameters": {
        "content": "No code behind these nodes, only a skeleton sample for the workflow could look like",
        "height": 208,
        "width": 944
      },
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        384,
        -304
      ],
      "typeVersion": 1,
      "id": "db35642a-04dd-427b-9a21-d4ad64dd5522",
      "name": "Sticky Note1"
    },
    {
      "parameters": {
        "content": "## Via data tabel\n",
        "height": 592,
        "width": 1088,
        "color": 3
      },
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        288,
        -384
      ],
      "typeVersion": 1,
      "id": "877a2308-c70f-4b9e-8ef7-5dea94a812b8",
      "name": "Sticky Note2"
    },
    {
      "parameters": {
        "content": "## Direct response from remote API",
        "height": 400,
        "width": 1088,
        "color": 5
      },
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        288,
        224
      ],
      "typeVersion": 1,
      "id": "d06d7c02-a8eb-494b-8d10-142fb2c04037",
      "name": "Sticky Note3"
    }
  ],
  "connections": {
    "Sample weather JSON data": {
      "main": [
        [
          {
            "node": "Convert json to array",
            "type": "main",
            "index": 0
          },
          {
            "node": "Manual mapping of one item",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Convert json to array": {
      "main": [
        [
          {
            "node": "Respond to Webhook",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Convert json to array1": {
      "main": [
        [
          {
            "node": "insert (or update if exists) table with data",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Sample weather JSON data1": {
      "main": [
        [
          {
            "node": "Convert json to array1",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Webhook - Respond after API call": {
      "main": [
        [
          {
            "node": "Sample weather JSON data",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Schedule to get weather data": {
      "main": [
        [
          {
            "node": "Sample weather JSON data1",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Webhook - respond with table data": {
      "main": [
        [
          {
            "node": "Get row(s)",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Get row(s)": {
      "main": [
        [
          {
            "node": "Edit Fields",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Edit Fields": {
      "main": [
        [
          {
            "node": "Respond to Webhook1",
            "type": "main",
            "index": 0
          }
        ]
      ]
    }
  },
  "pinData": {},
  "meta": {
    "instanceId": "139fc1c0283616d9fae3813879a161cf8e8e0992eea7a9e87dc980c7d4ed97ec"
  }
}
        • irmadlad@lemmy.worldOPEnglish
          2·
          2 months ago

          Oh Snap! You didn’t. I will definitely get this running in a bit when I finish all my chores and can go play. Wow man, thank you so much!! I had put this on my dashboard as a means of encouragement to finish what I started.

          spoiler

          ETA: That’s so purty. Haven’t fired it up, just imported. Got chores to do. Will report back.

          spoiler

        • irmadlad@lemmy.worldOPEnglish
          2·
          2 months ago

          Just a follow up to first say thank you, and that the script is working for me. I’m doing a bit of tweaking but it certainly jump started the process. You’re a scholar and a gentleman.

          • captcha_incorrect@lemmy.worldEnglish
            2·
            2 months ago

            Glad to hear that you got it working! The hard part for me was to figure out how to transform a JSON string into an array of n8n objects. I had already done that before so now I just copy that node whenever I need something like again.

            I’m doing a bit of tweaking

            I figured you would have. But that is half the fun. :-)