Starting Clean - Kibana
-
Filebeat is the forwarder for logstash. You could use both elasticsearch and logstash and not install kibana. It's just a front end for data visualization.
-
@DustinB3403 said in Starting Clean - Kibana:
Kibana is supposed to be using Elk, Logstash and Filebeat to collect and present the logs.
At least according the guide written.
Kibana doesn't "use" anything. I think you are confusing what the parts do. Kibana is just the interface on top, it just shows graphs and stuff. The system doing the work is Logstash and ElasticSearch. Logstash is using Filebeat. Logstash is storing the data. Logstash is the real application here. Kibana doesn't "do" anything when you aren't looking at it.
-
@DustinB3403 said in Starting Clean - Kibana:
Well then what is wrong here, I'm about fed up with trying to figure this Kibana out..
Figure out Logstash and Filebeat, the rest will take care of itself. RIght now, Filebeat isn't running. Start there. Why isn't it starting. Look at the logs.
-
Looking right so far?
-
Yup, it is up and running now. Now monitor the logs, it should tell you when log egress happens.
-
So in a sidebar conversation with @scottalanmiller
I don't have any new logs in /var/log on XS6.5
So where else should I look for this?
(Same on the logging server)
-
@DustinB3403 Did you use XC to change the logging to a remote location?
-
@Danp said in Starting Clean - Kibana:
@DustinB3403 Did you use XC to change the logging to a remote location?
Yes.
-
@DustinB3403 Then it stops writing to the local logs in some cases as described at the bottom of this article.
-
OK I'm done with this trial......
I'm just going in circles and before I break something I need a breather...
-
Damn - I'm glad Dustin ran through this first I think I would have been pulling my hair out LONG before he did.
Dustin (and I) want NO local logging on our XS boxes. We would love to have this log information inside something like ELK where we can do easy searches and graphs on it.
If Filebeat forwards the local logs to the ELK server how do we get a situation where no local logs are stored on the XS boxes?
Do we seriously have to setup a syslog server in the middle that does nothing but collect logs and run Filebeat, which then forwards the syslog's logs to the ELK server?
-
Hopefully there is a way to send the logs to Logstash directly, instead of sending them via Filebeat.
-
@DustinB3403 said in Starting Clean - Kibana:
@Danp said in Starting Clean - Kibana:
@DustinB3403 Did you use XC to change the logging to a remote location?
Yes.
Check the logs there, then.
-
@Dashrender said in Starting Clean - Kibana:
Hopefully there is a way to send the logs to Logstash directly, instead of sending them via Filebeat.
Of course, syslog, which you are already running. But Filebeat makes Logstash ingest easier.
-
@scottalanmiller said in Starting Clean - Kibana:
@DustinB3403 said in Starting Clean - Kibana:
@Danp said in Starting Clean - Kibana:
@DustinB3403 Did you use XC to change the logging to a remote location?
Yes.
Check the logs there, then.
I'm guessing there is not there - because the 'there' that he is forwarding them to is the ELK server.
-
@Dashrender said in Starting Clean - Kibana:
@scottalanmiller said in Starting Clean - Kibana:
@DustinB3403 said in Starting Clean - Kibana:
@Danp said in Starting Clean - Kibana:
@DustinB3403 Did you use XC to change the logging to a remote location?
Yes.
Check the logs there, then.
I'm guessing there is not there - because the 'there' that he is forwarding them to is the ELK server.
You can't both forward to ELK and use Filebeat!!
Of course this isn't working. -
@Dashrender said in Starting Clean - Kibana:
Hopefully there is a way to send the logs to Logstash directly, instead of sending them via Filebeat.
-
@scottalanmiller migth I ask you try doing this very same thing on an XS installation and outline exactly what you do so that others can replicate it.
As it is now, I'm beyond upset with just trying to get this going.
-
@Dashrender said
Dustin (and I) want NO local logging on our XS boxes.
The only way I ever got this to work was the "dirty little trick" that is mentioned in the comments of that article.
Because on mine, it kept writing locally REGARDLESS of what I did.
@scottalanmiller determined the best way was to move /var/logs
-
@scottalanmiller said in Starting Clean - Kibana:
@Dashrender said in Starting Clean - Kibana:
Hopefully there is a way to send the logs to Logstash directly, instead of sending them via Filebeat.
Of course, syslog, which you are already running. But Filebeat makes Logstash ingest easier.
You've lost me. you say of course, syslog, but Dustin took down the syslog server when he stood the ELK server up, because the assumption was/is that the ELK server replaced the syslog server.
