Jump to content
Chris Goluch

Custom Databases

Recommended Posts

Hi,

 

I am going to ask a few questions regarding widget development:

  1. what DB software do you use (mogo, mysql, plaintext)?
  2. what npm module is used to manage this database?
  3. with out using any thing like frog forms, frog data viewer, csv files, frog documents of any kind. Could I please get access to give my widgets an actual database table?
  4. Can we get the ability to read console print outs from the server side code execution of widgets?
  5. Can we please get some form of API access for the server side part of EJS templates, mainly for use with databases.

 

Essentially as somebody who has limited experience developing web apps, I find that the current widget system of having to hack in things from locally hosted files, or using the client to connect to a Document based database hosted on 365 is too janky. Client side access for rendering inside an EJS template is exactly what I want but I don't have the tools available to do the really cool stuff I want to do.

 

In a nut shell you can't watch Doctor Who on BBC one if you have a Television, but no TV channels. EJS is our television but right now we have no decent channels to watch.

 

I know, I make a post about this every year but, Frog Code seriously needs to start being worked on again, right now we can not even delete our demo widget projects. I think Frog Code needs a road map and a clear plan, or it needs to be removed if you have no plans on finishing this feature.

 

Sorry to sound like a brat, but I feel this is a seriously huge missed opportunity.

 

Thank you

Link to post
Share on other sites

Hi @Chris Goluch,

I've asked one of our developers to respond to your more technical questions, but I can answer the product roadmap-type ones.

Frog's front-end was designed around security.  No front-end tools have direct access to the databases - all access is through API requests.  This makes Frog very secure, for instance, you cannot send a "Delete Tables" query.  But it does mean that any data you wish to retrieve needs to have an API you can use to get it first.

Connecting to a 3rd-party database is awkward, because you need somewhere to store the secret keys, which is not possible without a major reworking of customer access to the data.  This was actually looked into, but the work involved was extensive and there just hasn't been that many schools asking for it.  It's a sad fact that there's not many @Chris Goluch in our schools.  We have to focus on features the majority will make use of.  Adding a database to FrogCode is a feature I would like to see, but we already have the Forms for secure data and the Data-Store.  both of these tools are accessible to FrogCode.  I'd be happy to share with you what I know about how to access and write to them.

We made the decision at the start of the year to look again at FrogCode, but then the world changed and priorities rightly shifted.  We do support it and maintain it, but the main focus is on stability.  I recommended leaving the Beta tag on the title for this reason.  It's a work-in-progress, just like the widgets it makes.

  • Thanks 1
Link to post
Share on other sites

Thanks for your response Graham.

I would love to get some technical details that is for sure.

In regards to the pandemic putting Frog code on the back burner again, I can understand that.

I would still love to see Frog Code get itself out of beta.

  • Like 1
Link to post
Share on other sites

Hi Chris, I'm a senior developer at Frog. Graham asked me to take a look at your questions.

The core of Frog Learn uses a MySQL database behind a custom API; we keep this locked up to internal development to ensure security and integrity of the data, so it's not available for use in Frog Code development.

We have a datastore API available to widget developers that allows you to store and work with server side data specific to your widgets, you can see the documentation for this at the following address on your Frog Learn instance: https://frog.broadway-academy.co.uk/app/docs/. This document is currently missing documentation for the the "create" endpoint, which I've outlined below:

POST to frog.broadway-academy.co.uk/api/fdp/2/datastore/create

Provide the following parameters; note these are similar to the "update" params.

target_uuid String The uuid against which you wish to associate the data (e.g. for a widget this.options.content_uuid)
data String The data you want to store
user_uuid String The uuid of the user that this data is associated with (defaults to current user if this param is not passed)
alias String

The alias for this data, this can be anything you want, and is to help you differentiate the data. For example: 'student.locker_number, student.random_data' max (255 chars)

Link to post
Share on other sites

Hi Simon,

 

I thank you for your time to explain this.

The idea of using a database and not the frog data store would allow for more complex interactions between other on site facilities.

 

Unfortunately the data store will not do the things I am looking into.

 

I think I might have something I wrote in node over the past few years that might be of interest. It is an SQL wrapper that turns databases into node js class objects and allows you to send full query's in json to the server from the client.

 

I would be happy to discuss this further if you are interested.

 

 

Link to post
Share on other sites

@simon brahan

 

Hi Simon, I think i have had a bit of a mad idea.

After noticing frog supports oAuth1.0 (can we get 2.0?) with a back end api.

Could I read from / write to, the datastore store from say a node js application.

I could do, some extremely interesting things if the answer is yes.

 

Secondly, @Graham Quince I would be very interested to take a second look at your project, because i may have a use for it.

 

Thank you both, for taking time.

Edited by Chris Goluch
Link to post
Share on other sites

Yes, Chris, you can use OAuth (either flavour) to communicate with FrogOS. We can add a client of either type to your Frog Learn instance through a support ticket; the client is sensitive so we can't do it through this forum. Once you have that you can create your own web app in any way you like and have it communicate with Frog Learn using the publicly available APIs listed in the documentation I provided above.

Thanks for the suggestion of an SQL wrapper. We use something similar internally; however is we were to make the interface available to third parties we wouldn't be able to ensure the security, integrity or performance of our back end.

Link to post
Share on other sites

Morning @simon brahan

So looking into this further, can i use the API with out using oAuth.

Say, I wanted to dump a datastore value for every student into a csv from a node js script at 12am every morning. So that another service can consume that csv and do what ever with it.

Otherwise to use oAuth means the user would need to log in first though a web client, its not that much of a problem just extra work.

But yes, I would love to have a play with oAuth if we can set that up. Are there any limits to the API and what impact would its use have on the frog box if any impact at all.

 

In regards to the SQL access, I would simply use a separate sql server. Using the one on the frog box would not be something that would work well long term for 3rd party databases.

 

 

Link to post
Share on other sites

You need to authenticate as a FrogOS Learn user to ensure your application isn't given access to permissions it shouldn't have. You should be able to log in as an admin user and then use most of Frog Learn's APIs, including the datastore ones.

There are some limits to how many API calls you make in a certain amount of time; you're unlikely to hit these as they're mainly to protect against denial of service or bot attacks which will send many requests very quickly. If you do hit them we can discuss changing the limits for your instance, or providing an API that will let you do the work in fewer calls.

Link to post
Share on other sites

Thank you again Simon.

 

I have now confused your support team with this request, but I have faith in them 🙂

 

I think I will let you in on my plans, maybe you will have input on this project idea.

My plan is to create a new server running a Bio-Store extractor and a my little node js webservice.

Every hour, the Bio-Store extractor will pipe out a csv of Bio-Store balances, this csv is then ingested by my node app. The API is then used to set a value in the data store for each user. That value can then be displayed later using a custom widget. 😊

I often export all the student data from frog and use a node to perform csv magic for user account management tasks. I think generally this could be come some sort of tool for scheduling custom imports and exports from the frog data store.

Any suggestions, ideas, concerns, criticisms welcome.

Link to post
Share on other sites

We have several third party integrations of varying complexity all working well with frog. A few examples:

1. Current printing balance pulled from our printer server. Runs on demand. Uses frog data store as a fall back in case print server becomes unavailable.

2. Who is absent today? Update for the frog seating plan just loaded pulled live from the sims server.

3. Haven’t done your register? An annoying but can’t fail to ignore frog notification if you haven’t done your register 20 minutes into each lesson. Again pulled live from sims server.

4. Attendance letter requests. One or more of 5 different letters can be requested for one or more students via a custom attendance portal in frog. The letter mail merge file is created in frog but as there is limited address/parental information in frog it queries sims on the fly to add the missing information before invoking word to start the merge.


My main piece of advice: for data going into frog consider using JSON format rather than csv. JavaScript natively understands this format. You can use a .js file as a container for this to defeat any cross origin issues.

Good luck with your idea.

  • Thanks 1
Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


  • Similar Content

    • By Graham Quince
      Here's a new FrogCode widget:
      https://www.frogeducation.com/community/frogcode/frogcode-showcase/visitor-log
      We worked with George Bligh from Greenshaw on this widget.  When someone views a page with this widget on, it records their visit.  Anyone with Edit and Manage rights gets a button which displays a list of all visitors, their profile, their children (if a parent), how many times they have visited the site and the time and date of their last visit.

      (This widget uses the datastore - a little known about feature of Frog's API.)
       
      Updates
      This latest version 2.2 includes a "copy to clipboard" button, which allows you to copy the whole table and paste it into a spreadsheet program like Excel.
      The second new feature includes the ability to alter the Datastore's reference.  So if you wish to reset the log, instead of deleting the widget and adding it again, you can now change the reference and the information will remain stored.
       
    • By Graham Quince
      The brilliant @Simon Law has built a new widget which adds a site's menu to a page, to save us all from having to do this manually using text widgets.
      You can download this from the showcase page:
      https://www.frogeducation.com/community/frogcode/frogcode-showcase/navigation
    • By Graham Quince
      When I posted the snippet to allow you to restyle of My Notes, I hadn't thought that it wasn't as straight forward to change the title of Phrase of the Day. This bit of CSS is the easiest way I've found of changing all Phrases of the Day, although if you combine it with the the technique(?) mentioned in the restyling My Notes, you can restyle individual widgets in the same way.
      <style> .widget_phrase-wrap .header { height:50px; overflow:hidden; } .widget_phrase-wrap .header h2::before { content: 'My own title \a'; white-space: pre; } </style>  
    • By ann
      Hi
      Is there any way I can get a nested page to be centred on a page?
      I have created a number of pages with three equal columns - each one has an text widget containing an icon image linked to various places. Each of these 3 column pages are then nested onto one page.
      It looks fine on a PC as all three columns show across the page  but on a phone it justifies to the left.upsetting the design. 
       
      Thanks
      Ann

    • By ann
      HI 
      I am trying to update a the widget image on a  widget I have created (well @Graham Quince created really) . I can't get the image to change.
      I have called it icon.png.. I have clicked on Update Image in the assets /  Icon.png section and it uploads the new image but does not use it. I have logged out and logged in again . The original image that was in place is still used when I look at the widget list,  but it has disappeared under assests/icon.png
      Ann
×
×
  • Create New...