Charts on dashboard in Manageiq Capablanca : scheme of generating content and fixing problem

20:54 4 Comments

 A lot of people, which use Manageiq Capablanca, faced with problem of charts on dashboard. In this post i will descibe you scheme of building charts and solution of problem with dashboard. Let's go!


Scheme of building content of charts



In generating of contents of charts all functions of model MiqWidget  are involved.
But the most interesting:
-   generate_content
- generate_report
- generate_report_result
In function generate_content, firstly function generate_report is calling and then generate_report_result is calling. So, we have content to build our charts. Then we go to MiqWidget::ChartContent
--
class MiqWidget::ChartContent < MiqWidget::ContentGeneration
  def generate(user_or_group)
    theme = user_or_group.settings.fetch_path(:display, :reporttheme) if user_or_group.kind_of?(User)

    # TODO: default reporttheme to MIQ since it doesn't look like we ever change it
    theme ||= "MIQ"
    report.to_chart(theme, false, MiqReport.graph_options)
    report.chart
  end
end

We can see, that all charts use standard graphic options(size, colours and other).
So, then we should to know more about function to_chart:

--
module MiqReport::Formatters::Graph
  extend ActiveSupport::Concern

  module ClassMethods
  ...
  def to_chart(theme = nil, show_title = false, graph_options = nil)
    ReportFormatter::ReportRenderer.render(Charting.format) do |e|
      e.options.mri           = self
      e.options.show_title    = show_title
      e.options.graph_options = graph_options unless graph_options.nil?
      e.options.theme         = theme
    end

  end
end
Charting.format = jqplot ( for all charts). But what about ReportFormatter::ReportRenderer.render?

--
module ReportFormatter
  class ReportRenderer < Ruport::Controller
    stage :document_header, :document_body, :document_footer
    finalize :document
    options { |o| o.mri = o.show_title = o.theme = o.table_width = o.alignment = o.graph_options = nil }
  end
end

How we can see, ReportFormatter::ReportRenderer use standard library ruport, which use module ChartCommon. And this module, finally, build contents of charts. Look at module ChartCommon and some important functions of it:
-
    def build_document_body
      return no_records_found_chart if mri.table.nil? || mri.table.data.blank?

      # find the highest chart value and set the units accordingly for large disk values (identified by GB in units)
      maxcols = 8
      divider = 1
      if graph_options[:units] == "GB" && !graph_options[:composite]
        maxval = 0
        mri.graph[:columns].each_with_index do |col, col_idx|
          next if col_idx >= maxcols
          newmax = mri.table.data.collect { |r| r[col].nil? ? 0 : r[col] }.sort.last
          maxval = newmax if newmax > maxval
        end
        if maxval > 10.gigabytes
          divider = 1.gigabyte
        elsif maxval > 10.megabytes
          graph_options[:units] = "MB"
          divider = 1.megabyte
        elsif maxval > 10.kilobytes
          graph_options[:units] = "KB"
          divider = 1.kilobyte
        else
          graph_options[:units] = "Bytes"
          graph_options[:decimals] = 0
          divider = 1
        end
        mri.title += " (#{graph_options[:units]})" unless graph_options[:units].blank?
      endfun = case graph_options[:chart_type]
            when :performance then :build_performance_chart # performance chart (time based)
            when :util_ts     then :build_util_ts_chart     # utilization timestamp chart (grouped columns)
            when :planning    then :build_planning_chart    # trend based planning chart
            else                                            # reporting charts
              mri.graph[:mode] == 'values' ? :build_reporting_chart_numeric : :build_reporting_chart
            end
      method(fun).call(maxcols, divider)
    end
    
    def build_reporting_chart_other
      save_key   = nil
      counter    = 0
      categories = []                      # Store categories and series counts in an array of arrays
      mri.table.data.each_with_index do |r, d_idx|
        if d_idx > 0 && save_key != r[mri.sortby[0]]
          save_key = nonblank_or_default(save_key)
          categories.push([save_key, counter])    # Push current category and count onto the array
          counter = 0
        end
        save_key = r[mri.sortby[0]]
        counter += 1
      end
      # add the last key/value to the categories and series arrays
      save_key = nonblank_or_default(save_key)
      categories.push([save_key, counter])        # Push last category and count onto the array

      categories.sort! { |a, b| b.last <=> a.last }
      (keep, show_other) = keep_and_show_other
      if keep < categories.length                      # keep the cathegories w/ highest counts
        other = categories.slice!(keep..-1)
        ocount = other.reduce(0) { |a, e| a + e.last } # sum up and add the other counts
        categories.push(["Other", ocount]) if show_other
      end

      series = categories.each_with_object(
        series_class.new(pie_type? ? :pie : :flat)) do |cat, a|
        a.push(:value => cat.last, :tooltip => "#{cat.first}: #{cat.last}")
      end

      # Pie charts put categories in legend, else in axis labels
      limit = pie_type? ? LEGEND_LENGTH : LABEL_LENGTH
      categories.collect! { |c| slice_legend(c[0], limit) }
      add_axis_category_text(categories)
      add_series(mri.headers[0], series)
    end 

Functions of module ChartCommon form content of charts belonging to MiqWidget, and this content is saved in database in table miq_widgets_contents.
Then, when we want to see at charts on dashboard this content is parsed by YAMl parser and converted in charts. Look at page of charts:

-
-#
  Parameters:
    widget -- MiqWidget instance
- width  ||= 350
- height ||= 250
.panel-body.chart_widget{:style => "text-align: center; padding: 0"}
  .mc{:id => "dd_w#{widget.id}_box",
    :style => @sb[:dashboards][@sb[:active_db]][:minimized].include?(widget.id) ? 'display:none' : ''}

    - if widget.contents_for_user(current_user).contents.blank?
      = _('No chart data found')
      \. . .
    - datum = widget.contents_for_user(current_user).contents
    - if Charting.data_ok?(datum)
      -# we need to count all charts to be able to display multiple
      -# charts on a dashboard screen

      - datum = widget.contents_for_user(current_user).contents
      - WidgetPresenter.chart_data.push(:xml => datum)
      - chart_index = WidgetPresenter.chart_data.length - 1
      - chart_data = YAML.load(datum) if Charting.backend == :jqplot

      = chart_local(chart_data,
                    :id     => "miq_widgetchart_#{chart_index}".html_safe,
                    :width  => width,
                    :height => height)
    - else
      = _('Invalid chart data. Try regenerating the widgets.')
 

Problem with charts and its solution

When you install new version of Manageiq - Capablanca, you can see this on dashboard:


The reason for this problem is that in ChartCommon data is generated
in the form of Hash, but we need YAML.
So, you should add one correction in MiqWidget::ChartContent:

-
class MiqWidget::ChartContent < MiqWidget::ContentGeneration
  def generate(user_or_group)
    theme = user_or_group.settings.fetch_path(:display, :reporttheme) if user_or_group.kind_of?(User)

    # TODO: default reporttheme to MIQ since it doesn't look like we ever change it
    theme ||= "MIQ"
    report.to_chart(theme, false, MiqReport.graph_options)
    report.chart.to_yaml
  end
end

 
Result:


Unknown

IBA Group, Minsk

4 comments:

  1. Hi,

    Where can I find the code in ManageIQ?

    Zoli

    ReplyDelete
  2. Replies
    1. Where did you find it ?
      Can you give me a hint ?

      Delete
  3. Thanks for the nice dive-in article! Really nice insight into the logic behind the charts.

    Fix in upstream by @himdel is here https://github.com/ManageIQ/manageiq/commit/3876720f

    ReplyDelete