ML
    • Recent
    • Categories
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    Starting Clean - Kibana

    Scheduled Pinned Locked Moved IT Discussion
    68 Posts 6 Posters 6.3k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • scottalanmillerS
      scottalanmiller
      last edited by

      Where is Filebeat running? Did you make sure that it is running? If you look at the logs (always /var/log/) does it say that Filebeat is working? Filebeat logs well.

      1 Reply Last reply Reply Quote 0
      • DustinB3403D
        DustinB3403
        last edited by

        Filebeat is installed on XS.

        0_1471370200678_putty_2016-08-16_13-56-29.png

        DanpD stacksofplatesS scottalanmillerS 3 Replies Last reply Reply Quote 0
        • DanpD
          Danp @DustinB3403
          last edited by

          @DustinB3403 said in Starting Clean - Kibana:

          Filebeat is installed on XS.

          Are you sure?! That looks like the info for the current executing command.

          1 Reply Last reply Reply Quote 1
          • stacksofplatesS
            stacksofplates @DustinB3403
            last edited by

            @DustinB3403 said in Starting Clean - Kibana:

            Filebeat is installed on XS.

            0_1471370200678_putty_2016-08-16_13-56-29.png

            No that's the grep you ran for Filebeat. It's not running.

            1 Reply Last reply Reply Quote 1
            • scottalanmillerS
              scottalanmiller @DustinB3403
              last edited by

              @DustinB3403 said in Starting Clean - Kibana:

              Filebeat is installed on XS.

              0_1471370200678_putty_2016-08-16_13-56-29.png

              ps tells you what is running, not what is installed (although if it is running, it would imply that it is installed.)

              Yours shows that it is not running. But grep is running 🙂

              Old timer Linux admin trick, tack this onto your ps commands....

              ps aux | grep filebeat | grep -v grep
              

              It greps out grep.

              1 Reply Last reply Reply Quote 1
              • DustinB3403D
                DustinB3403
                last edited by

                Well then what is wrong here, I'm about fed up with trying to figure this Kibana out..

                scottalanmillerS 1 Reply Last reply Reply Quote 0
                • stacksofplatesS
                  stacksofplates
                  last edited by

                  Use graylog and rsyslog. If you are supposed to treat XenServer as an appliance, don't install Filebeat and use the built in tools.

                  1 Reply Last reply Reply Quote 0
                  • stacksofplatesS
                    stacksofplates
                    last edited by

                    Also Filebeat is for logstash not kibana.

                    1 Reply Last reply Reply Quote 0
                    • DustinB3403D
                      DustinB3403
                      last edited by

                      Kibana is supposed to be using Elk, Logstash and Filebeat to collect and present the logs.

                      At least according the guide written.

                      scottalanmillerS 1 Reply Last reply Reply Quote 0
                      • stacksofplatesS
                        stacksofplates
                        last edited by

                        Filebeat is the forwarder for logstash. You could use both elasticsearch and logstash and not install kibana. It's just a front end for data visualization.

                        1 Reply Last reply Reply Quote 0
                        • scottalanmillerS
                          scottalanmiller @DustinB3403
                          last edited by

                          @DustinB3403 said in Starting Clean - Kibana:

                          Kibana is supposed to be using Elk, Logstash and Filebeat to collect and present the logs.

                          At least according the guide written.

                          Kibana doesn't "use" anything. I think you are confusing what the parts do. Kibana is just the interface on top, it just shows graphs and stuff. The system doing the work is Logstash and ElasticSearch. Logstash is using Filebeat. Logstash is storing the data. Logstash is the real application here. Kibana doesn't "do" anything when you aren't looking at it.

                          1 Reply Last reply Reply Quote 1
                          • scottalanmillerS
                            scottalanmiller @DustinB3403
                            last edited by

                            @DustinB3403 said in Starting Clean - Kibana:

                            Well then what is wrong here, I'm about fed up with trying to figure this Kibana out..

                            Figure out Logstash and Filebeat, the rest will take care of itself. RIght now, Filebeat isn't running. Start there. Why isn't it starting. Look at the logs.

                            1 Reply Last reply Reply Quote 0
                            • DustinB3403D
                              DustinB3403
                              last edited by

                              Looking right so far?

                              0_1471372534761_putty_2016-08-16_14-35-28.png

                              1 Reply Last reply Reply Quote 0
                              • scottalanmillerS
                                scottalanmiller
                                last edited by

                                Yup, it is up and running now. Now monitor the logs, it should tell you when log egress happens.

                                1 Reply Last reply Reply Quote 0
                                • DustinB3403D
                                  DustinB3403
                                  last edited by DustinB3403

                                  So in a sidebar conversation with @scottalanmiller

                                  I don't have any new logs in /var/log on XS6.5

                                  0_1471373503538_putty_2016-08-16_14-51-37.png

                                  So where else should I look for this?

                                  (Same on the logging server)

                                  DanpD 1 Reply Last reply Reply Quote 0
                                  • DanpD
                                    Danp @DustinB3403
                                    last edited by

                                    @DustinB3403 Did you use XC to change the logging to a remote location?

                                    DustinB3403D 1 Reply Last reply Reply Quote 1
                                    • DustinB3403D
                                      DustinB3403 @Danp
                                      last edited by

                                      @Danp said in Starting Clean - Kibana:

                                      @DustinB3403 Did you use XC to change the logging to a remote location?

                                      Yes.

                                      DanpD scottalanmillerS 2 Replies Last reply Reply Quote 0
                                      • DanpD
                                        Danp @DustinB3403
                                        last edited by

                                        @DustinB3403 Then it stops writing to the local logs in some cases as described at the bottom of this article.

                                        1 Reply Last reply Reply Quote 1
                                        • DustinB3403D
                                          DustinB3403
                                          last edited by

                                          OK I'm done with this trial......

                                          I'm just going in circles and before I break something I need a breather...

                                          1 Reply Last reply Reply Quote -1
                                          • DashrenderD
                                            Dashrender
                                            last edited by

                                            Damn - I'm glad Dustin ran through this first I think I would have been pulling my hair out LONG before he did.

                                            Dustin (and I) want NO local logging on our XS boxes. We would love to have this log information inside something like ELK where we can do easy searches and graphs on it.

                                            If Filebeat forwards the local logs to the ELK server how do we get a situation where no local logs are stored on the XS boxes?

                                            Do we seriously have to setup a syslog server in the middle that does nothing but collect logs and run Filebeat, which then forwards the syslog's logs to the ELK server?

                                            BRRABillB 1 Reply Last reply Reply Quote 0
                                            • 1
                                            • 2
                                            • 3
                                            • 4
                                            • 1 / 4
                                            • First post
                                              Last post